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1.  INTRODUCTION  AND  SUMMARY 


The  Autonomous  Landing  Guidance  (ALG)  Program  is  an  effort  aimed  at  providing 
commercial  and  military  transports  with  the  capability  to  perform  approaches  and  landings  in 
low-visibility  conditions  at  airfields  that  are  not  equipped  for  such  operations.  An  industry- 
government  alliance  was  formed  to  develop  an  ALG  prototype  for  transport  aircraft.  In 
support  of  the  alliance,  the  Advanced  Cockpits  Branch  of  Wright  Laboratory  (WL/FIGP) 
conducted  a  study  that  will  support  the  definition  of  head-up  display  (HUD)  symbology  and 
sensor  imagery  integration  requirements  for  ALG  applications  in  commercial  and  military 
transport  aircraft.  The  backgroimd  and  objectives  of  this  study  are  described  in  this  section,  as 
well  as  a  summary  of  the  methodology  used  and  the  study’s  findings. 

1.1  Background 

The  concept  of  ALG  systems  is  to  use  existing  technologies,  including  HUDs,  the  Global 
Positioning  System  (GPS),  forward  looking  infra-red  (FLIR)  sensor  imagery,  and  millimeter 
wave  radar  (MMWR)  sensor  imagery  to  augment  traditional  landing  aids  in  low-visibility 
conditions.  The  goal  of  the  ALG  program  is  to  substantially  expand  low- visibility  operations 
for  both  commercial  and  military  aircraft.  Consequently,  the  successful  development  of  an 
ALG  system  will  also  provide  greater  operational  flexibility  for  military  operations. 
Specifically,  an  ALG  system  ■will  allow  special  and  tactical  military  operations  to  venture  into 
austere  or  unprepared  landing  fields  regardless  of  the  weather  conditions. 

In  the  case  of  transport  aircraft,  the  Federal  Aviation  Administration  (FAA)  defines  five 
weather  categories  associated  Avith  low-visibility  approach  conditions  using  minimum  ceiling 
and  runway  visibility  values.  Current  regulations  (Federal  Aviation  Regulations/ Airman’s 
Information  Manual  (FAR/AIM))  limit  Category  (CAT)  II  and  CAT  Ilia’  approaches  and 


‘  CAT  II  minimums:  >1200  ft  runway  visibility  range  (RVR),  100  ft  decision  height;  CAT  Ilia 
minimums:  0  through  700  ft  RVR,  no  decision  height. 
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landings  to  runways  that  conform  to  strict  Instrument  Landing  System  (ILS),  lighting,  and 
obstruction  clearance  requirements.  Because  it  is  expensive  to  equip  and  maintain  airfields  for 
CAT  II  and  CAT  III  operations,  the  vast  majority  of  airfields  are  equipped  for  CAT  I 
operations  (>1800  ft  RVR,  200  ft  decision  height).  Very  few  CAT  II  or  CAT  III  facilities 
currently  exist  and  the  assumption  is  that  few  will  be  constructed.  The  use  of  an  ALG  system 
will  allow  both  commercial  and  military  aircraft  to  safely  land  at  airfields  equipped  for  CAT  I 
operations  in  visibility  conditions  below  the  CAT  I  minimums. 

1.2  Objectives 

In  an  aircraft  equipped  for  ALG,  GPS-driven  flight  and  approach  guidance  symbology 
and  a  sensor-generated  image  of  the  runway  environment  will  be  provided  on  a  HUD,  with  the 
symbology  superimposed  over  the  sensor  imagery.  During  a  low-visibility  precision  approach 
with  an  ALG  system,  the  pilot  will  use  traditional  navigation  instruments  to  perform  the  initial 
approach,  the  GPS-driven  HUD  symbology  throughout  the  approach,  and  the  visual  cues  from 
the  sensor  image,  rather  than  natural  vision  to  perform  a  final  landing  decision. 

Before  an  ALG  HUD  can  serve  as  an  effective  pilot-vehicle  interface  (PVI),  the 
symbology  and  the  sensor  imagery  must  provide  the  critical  information  required  for  a  low- 
visibility  precision  approach.  In  addition,  the  HUD  symbology  must  not  unnecessarily  clutter 
the  display  or  obscure  the  important  visual  cues  provided  by  the  sensor  imagery. 
Occasionally,  these  two  requirements  conflict.  The  addition  of  symbology  may  be  desirable 
to  provide  important  flight  information,  but  the  addition  of  the  symbology  may  also  contribute 
to  clutter  and  masking  of  the  sensor  image.  The  HUD  symbology  requirements  may  also 
depend  on  the  quality  of  the  sensor  image.  WL/FIGP  is  addressing  these  issues  through  its 
cockpit  research  program.  The  three  objectives  of  the  ALG  study  were  as  follows: 
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•  Objective  1 

Determine  the  relative  effects  of  different  symbology  sets  and  their  inherent  levels 
of  clutter  on  pilot  performance,  pilot  acceptance,  and  pilot  workload. 

•  Objective  2 

Determine  the  effect  of  sensor  imagery  on  pilot  performance,  pilot  acceptance,  and 
pilot  workload  in  an  ALG  context. 

•  Objective  3 

Identify  “safe  minimum”  and  “optimum”  ALG  HUD  symbology  sets. 

1.3  Scope 

This  report  describes  the  methodology  and  results  of  the  study  completed  by  WL/FIGP  in 
support  of  the  development  effort  for  an  ALG  system.  It  also  recommends  HUD  information 
and  symbology  requirements  for  an  ALG  system  that  are  based  on  pilot  performance,  pilot 
workload,  and  pilot  preferences.  Finally,  the  report  addresses  requirements  and  considerations 
for  future  research.  Study  findings  are  presented  to  provide  design  guidance  to  ALG  system 
designers  and  integrators,  as  well  as  to  provide  guidance  for  failure  mode  analyses. 

1.4  Executive  Summary 

To  accomplish  the  study’s  objectives,  12  pilots  performed  a  flight  simulation  experiment, 
a  HUD  assembly  experiment,  and  completed  a  questionnaire.  In  the  flight  simulation 
experiment,  the  pilots  flew  a  series  of  precision  approaches  (referred  to  as  the  Precision 
Approach  Task)  using  four  experimental  HUD  Symbology  Sets  in  four  SensorA^isibility 
Conditions.  Pilot  performance,  pilot  workload,  and  pilot  acceptance  ratings  were  collected 
during  each  precision  approach.  The  four  experimental  HUD  Symbology  Sets  were:  1)  a 
symbology  set  that  included  a  nmway  representation  (the  Basic  Plus  Runway  Sjnnbology  Set 
or  Basic/RWY);  2)  a  set  that  included  raw  deviation  data  (the  Basic  Plus  Raw  Deviation  Data 
or  Basic/RAW);  3)  a  set  that  had  both  raw  data  and  a  runway  symbol  (Basic/BOTH);  and  4) 
a  set  composed  of  the  full  complement  of  symbology  (the  Full  HUD).  The  four  HUD 
Symbology  Sets  used  in  the  study  were  based  on  the  Sextant  Avionique  HUD  format,  with 
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each  containing  the  minimum  symbology  required  for  basic  instrument  flying  (airspeed, 
altitude,  vertical  velocity,  flight  path/pitch  ladder,  heading,  and  bank  angle). 

The  four  SensorA^isibility  Conditions  represented  various  levels  of  visual  image  quality 
and  were:  1)  Visual  Meteorological  Conditions  (VMC)  providing  an  unlimited  visibility,  full 
field-of-view  (FOV),  daytime  scene;  2)  Instrument  Meteorological  Conditions  (IMC)  with  a 
25°  X  35°  FOV  FLIR  image  on  the  HUD;  3)  a  Comer  Reflector  Condition  representing  a 
MMWR  image  of  a  series  of  comer  reflectors  outlining  the  runway;  and  4)  a  No  Sensor 
Condition  representing  CAT  Ilia  weather  minimums  with  a  0  ft  decision  height  and  700  ft 
RVR.  In  the  Precision  Approach  Task,  the  pilots  flew  each  combination  of  HUD  Symbology 
Set  and  SensorA^isibility  Condition  twice,  totaling  32  approaches. 

In  the  HUD  assembly  experiment,  the  pilots  deflned  HUD  symbology  sets  that  would  be 
required  to  complete  a  precision  approach  under  a  variety  of  Approach  and  SensorA^isibility 
Conditions.  For  each  condition,  the  pilots  identified  two  sets  of  symbols.  The  first  was  the 
minimum  symbology  set  the  pilots  thought  would  allow  the  safe  completion  of  an  approach. 
The  second  set  included  the  minimum  safe  set  and  additional  symbols  the  pilots  selected  to 
optimize  the  format.  After  the  pilots  finished  both  experiments,  they  completed  a 
questionnaire  (see  Appendix  A)  that  included  a  workload  evaluation,  as  well  as  a  variety  of 
questions  and  rating  scales  regarding  the  HUD  Symbology  Sets.  The  data  were  used  to 
compare  pilot  workload  across  the  test  conditions  and  to  aid  interpretation  of  the  performance 
data. 

WL/FIGP  conducted  the  experiments  in  the  Transport  Aircraft  Cockpit  (TRAC)  flight 
simulator,  a  fully  reconfigurable  research  simulator  containing  two  crew  positions  (pilot  and 
co-pilot)  and  a  third  station  for  the  test  engineer.  Wide-angle  collimated  (WAC)  displays 
mounted  directly  in  front  of  each  crew  position  dynamically  displayed  the  HUD  Symbology 
Sets  and  SensorWisibility  Conditions. 
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Analyses  of  Variance  (ANOVA)  and  the  appropriate  post-hoc  tests  were  applied  to  the 
performance  and  workload  data  collected  in  the  Precision  Approach  Task.  For  the  subjective 
data,  including  the  symbology  selections  from  the  HUD  Assembly  Task,  frequency 
distributions  and  means  were  calculated  and  comments  were  summarized. 

During  the  precision  approaches,  the  pilots  performed  least  accurately  and  rated  workload 
highest  with  the  Basic/RWY  (the  least  amount  of  approach  information)  and  the  Full  HUD 
(the  greatest  amount  of  approach  information)  Symbology  Sets.  Conversely,  pilots  performed 
most  accurately  and  rated  workload  lowest  for  the  Basic/RAW  and  Basic/BOTH  Symbology 
Sets.  On  the  questionnaires,  the  pilots  commented  that  the  Basic/RWY  did  not  provide 
adequate  information  to  safely  complete  the  approach,  and  that  the  Full  HUD  presented  too 
much  information  and  was  too  cluttered.  The  clutter  and  high  symbology  density  associated 
with  the  Full  HUD  caused  distraction  and  obscured  the  sensor/visual  image.  Finally,  the 
pilots  landed  closer  to  the  designated  touchdown  point  in  the  VMC  and  FLIR  Conditions,  than 
they  did  in  the  Comer  Reflector  or  No  Sensor  Conditions. 

In  the  HUD  Assembly  Task,  the  pilots  identified  symbology  sets  resembling  the 
Basic/RAW  Symbology  Set  for  both  the  minimum  and  optimum  formats.  With  the  exception 
of  groundspeed  and  the  aircraft  reference  symbol,  the  pilots  retained  all  of  the  symbology 
from  the  Basic  format.  For  the  minimum  set,  more  than  half  the  pilots  added  a  selected  course 
indicator,  the  digital  distance  measuring  equipment  (DME),  and  course  deviation  symbology 
to  the  Basic  Symbology  Set.  For  the  optimum  set,  more  than  half  the  pilots  selected  the 
symbols  included  in  the  minimum  set  and  added  a  flight  director  for  the  long  final  situations. 
They  added  a  wind  vector  symbol,  a  rising  runway,  and  the  perspective  runway  symbol  for  the 
short  final  situations.  These  results  are  consistent  with  the  findings  of  the  performance  and 
workload  analyses. 

The  recommended  optimum  HUD  Symbology  Sets  provide  design  guidelines  for  ALG 
system  designers  that  are  consistent  with  the  HUD  symbology  requirements  and 
recorrunendations  identified  in  military  standards  and  previous  research  (MIL-STD-1787; 
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Todd  et  al.,  1993).  The  minimum  safe  set  provides  a  starting  point  for  defining  declutter 
options.  The  findings  also  indicated  that  in  low-visibility  situations,  sensor  imagery  of 
sufficient  quality  presented  on  the  HUD  will  elicit  performance  that  approaches  that  achieved 
inVMC. 
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2.  THE  HUD  SYMBOLOGY  SETS 


Pilots  flew  four  different  Symbology  Sets  in  the  evaluation:  1)  the  Basic  Plus  Runway 
Symbology  Set,  2)  the  Basic  Plus  Raw  Deviation  Data  Symbology  Set,  3)  the  Basic  Plus  Both 
Runway  and  Raw  Deviation  Data  Symbology  Set,  and  4)  the  Full  Head-Up  Display  (HUD). 
All  of  the  sets  were  variations  of  the  Sextant  Avionique  HUD  format  (Coirier,  1995).  The 
Sextant  format  is  currently  used  only  in  commercial  aircraft  applications,  but  is  representative 
of  the  information  content  and  formats  currently  implemented  in  military  transport  aircraft. 
This  study  was  not  intended  to  be  a  critical  evaluation  of  the  Sextant  HUD,  but  rather  an 
information  requirements  definition  exercise  with  the  Sextant  HUD  serving  as  the  test 
gyjjibology  set.  The  Sextant  HUD  is  similar  to  other  HUDs  and  was  chosen  due  to  its 
exposure  within  the  Autonomous  Landing  Guidance  (ALG)  industry-government  alliance  and 
the  availability  of  the  specifications  necessary  to  incorporate  its  symbology  into  the  Transport 
Aircraft  Cockpit  (TRAC)  simulator. 

The  test  Symbology  Sets  included  basic  symbology  that  provided  information  required 
for  instrument  flight,  including  a  flight  path  marker  and  scale,  pitch,  bank,  airspeed, 
barometric  altitude,  radar  altitude,  ground  speed,  vertical  velocity,  and  heading.  This  symbol 
set,  referred  to  as  the  “Basic  Symbology  Set,”  conforms  to  the  military’s  primary  flight 
display  requirements  for  instrument  flying  (M1L-STD-1787B).  The  test  Symbology  Sets  were 
selected  based  on  an  analysis  of  the  information  requirements  associated  with  a  precision 
approach  and  the  differences  in  the  quantity  and  precision  of  the  approach  information  they 
provided. 

The  following  sub-sections  describe  the  test  Symbology  Sets  and  include  an  image  of 
each  as  obtained  from  the  simulation.  Each  of  the  individual  symbols  can  be  identified  using 
the  representative  schematic  of  the  HUD  (see  Figure  1).  The  schematic  (Figure  1)  provides  an 
identity  to  each  of  the  HUD  symbols  discussed  in  this  report. 
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a.  Aircraft  Reference 

b.  Horizon  Line 

c.  Vertical  Scale 

d.  Digital  Barometric  Altitude 

e.  Digital  Radar  Altitude 

/  Digital  Airspeed  (Actual) 
g.  Groundspeed 
k  Heading  Scale 

i.  Digital  Heading 

j.  Actual  Heading  Marker 


k.  Selected  Heading  Marker 
/.  Roll  Scale 

m.  Actual  Track  Marker 

n.  Digital  Vertical  Speed 

o.  Selectable  Vertical  Reference 

p.  Flight  Path  Marker 
g.  Runway 

r.  Raw  Azimuth  Deviation 

s.  Raw  Elevation  Deviation 

t.  Selected  Course 


M,  Speed  Error  Tape 
V.  Acceleration  Cue 
w.  Flight  Path  Guidance  Cue 
X.  Distance  Measuring  Equipment 

y.  Wind  Vector 

z.  Flare  Annunciator 
aa.  Landing  Advisory 
bb.  Flight  Yaw  Indicator 
cc.  Rising  Runway 


Figure  L  HUD  Symbology  Schematic 

2,1  The  Basic  Plus  Runway  Symbology  Set 

The  Basic  Plus  Runway  Symbology  Set  (Basic/RWY),  as  shown  in  Figure  2,  included  the 
Basic  Symbology  Set  and  a  perspective  runway  outline,  providing  the  approach  guidance. 
The  runway  symbol  dynamically  provided  gross  lateral  and  vertical  guidance  and  gross 
distance  to  the  runway,  which  was  based  on  the  position  and  orientation  of  the  aircraft  with 
respect  to  the  airstrip.  In  theory,  the  runway  symbol  is  intended  to  provide  the  same  visual 
cues  (e.g.,  perspective  and  orientation  information)  as  a  visual  scene  of  the  runway 
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environment.  Previous  researchers  (Todd,  et  al.,  1993)  suggested  that  a  perspective  runway 
symbol  may  be  able  to  serve  as  an  intuitive  substitute  for  traditional  HUD  approach 
symbology,  including  raw  deviation  data  and  flight  director  information.  In  this  study,  the 
runway  symbol  represented  the  minimum  approach  symbology  provision  as  it  provided  the 
least  and  lowest-precision  approach  information.  The  Basic/RWY  Symbology  Set  also  had 
the  least  symbology  and,  consequently,  the  least  clutter. 


Figure  2.  The  Basic  Plus  Runway  HUD  Symbology  Set 
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2.2  The  Basic  Plus  Raw  Deviation  Data  Symbology  Set 


The  Basic  Plus  Raw  Deviation  Data  Symbology  Set  (Basic/RAW),  as  shown  in  Figure  3, 
included  the  Basic  Symbology  Set  along  with  raw  azimuth  and  glideslope  deviation  symbols. 
The  runway  symbol  was  not  part  of  this  symbology  set.  This  set  included  approach 
symbology  that  has  traditionally  been  provided  by  head-up  and  head-down  display  formats. 
Relative  to  the  other  Symbology  Sets,  the  Basic/RAW  Symbology  Set  consisted  of  a 
moderate  quantity  of  symbology,  a  moderate  level  of  clutter,  and  high-precision  approach 
information. 


Figure  3.  The  Basic  Plus  Raw  Deviation  Data  HUD  Symbology  Set 


2.3  The  Basic  Plus  Both  Runway  and  Raw  Deviation  Data  Symbology  Set 

The  Basic  Plus  Both  Runway  and  Raw  Deviation  Data  Symbology  Set  (Basic/BOTH) 
was  identical  to  the  Basic/RAW  Symbology  Set,  except  for  the  addition  of  the  perspective 
runway,  as  shown  in  Figure  4.  It  provided  both  the  traditional,  high-precision  approach 
information,  as  well  as  the  intuitive  visual  cues  provided  by  the  runway  symbol  in  the  same 
format.  Because  it  provided  both  symbols,  the  Basic/BOTH  HUD  included  slightly  more 
symbology  and  clutter  than  the  Basic/RWY  and  the  Basic/RAW  Symbology  Sets. 


Figure  4.  The  Basic  Plus  Both  Runway  &  Raw  Deviation  Data  HUD  Symbology  Set 
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2.4  The  Full  Sextant  HUD 


The  Full  Sextant  HUD  (Full  HUD)  included  every  symbol  identified  in  the  Sextant 
specification  (Coirier,  1993).  The  Full  HUD  was  included  in  the  evaluation  to  represent  the 
condition  in  which  maximum  information  is  available.  The  Full  HUD  is  shown  in  Figure  5, 
and  the  HUD  Symbology  Sets  and  their  respective  symbols  are  summarized  in  Table  1.  Some 
of  the  symbols  in  Table  1  (those  accompanied  by  an  asterisk)  are  not  shown  in  the  HUD 
schematic  (Figure  1)  or  any  of  the  HUD  Symbology  Set  images  (Figures  2,  3,  4,  and  5). 
Because  they  were  either  mode  status  indicators  or  symbols  identifying  extreme  situations, 
these  symbols  were  rarely,  if  ever,  displayed.  As  such,  they  had  little  or  no  relevance  in  this 
study  and  were  not  formally  evaluated. 


Figure  5.  The  Full  Sextant  HUD  Symbology  Set 
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Table  1.  The  HUD  Symbology  Sets 


HUD  Symbology  Elements 

Basic/RWY 

Basic/RAW 

Basic/BOTH 

Full  HUD 

Aircraft  Reference 

X 

X 

X 

X 

Horizon  Line  &  Vertical  Scale 

X 

X 

X 

X 

Digital  Barometric  and  Radio 
Altitudes 

X 

X 

X 

X 

Digital  Airspeed  (actual)  and 
Groundspeed 

X 

X 

X 

X 

Heading  Scale  with  Digital 
Heading,  Actual  and  Selected 
Heading  Markers 

X 

X 

X 

X 

Roll  Scale 

X 

X 

X 

X 

Actual  Track  Marker 

X 

X 

X 

Digital  Vertical  Speed 

Flight  Path  Marker 

X 

X 

X 

X 

X 

X 

Runway 

X 

X 

X 

Raw  Azimuth  Deviation 

X 

X 

X 

Raw  Elevation  Deviation 

X 

X 

X 

Selected  Course 

X 

X 

X 

Speed  Error  Tape  &  Acceleration 
Cue 

X 

Vertical,  Horizontal,  &  Steady 
Flight  Director  Mode  * 

X 

Flight  Path  Guidance  Cue 

X 

Digital  Distance  Measuring 
Equipment  (DME) 

X 

Wind  Vector 

X 

Course  Markers  * 

X 

Retard  &  Flare  Annunciators 

X 

Roll  &  Angle  of  Attack  Limits  * 

X 

Landing  Advisory 

X 

Flight  Yaw  Indicator 

X 

Selectable  Vertical  Reference 

X 

EVS  Status  * 

X 

Over  Flare  Caution  * 

X 

Centerline  Lateral  Guidance  * 

X 

To/From  Indicator  * 

X 

CDI  Needle  * 

X 

Rising  Runway 

X 

3.  THE  SENSOIWISIBILITY  CONDITIONS 


The  pilots  flew  each  head-up  display  (HUD)  Symbology  Set  in  four  computer  generated 
SensorA^isibility  Conditions:  1)  Visual  Meteorological  Conditions  (VMC),  2)  Instrument 
Meteorological  Conditions  (IMC)  with  forward  looking  infrared  (FLIR)  Sensor,  3)  IMC  with 
millimeter  wave  radar  (MMWR)  Sensor,  and  4)  IMC  with  No  Sensor.  These  SensorA^isibility 
Conditions  were  displayed  as  out-the-window  (OTW)  scenes  on  the  Transport  Aircraft 
Cockpit  (TRAC)  simulator’s  wide-angle  collimated  (WAC)  windows,  along  with  the  HUD 
symbology.  The  four  SensorA^isibility  Conditions  represented  baseline  visibility,  as  well  as 
two  levels  of  sensor  image  quality.  The  two  sensor  conditions  reflected  characteristic  sensor 
performance  requirements  to  which  Autonomous  Landing  Guidance  (ALG)  systems  are  being 
designed.  In  all  of  the  SensorA/isibility  Conditions,  the  images  were  detailed  representations 
of  two  actual  airfields  located  at  US  Air  Force  Bases  (AFBs):  Pope  AFB  (used  exclusively  in 
the  training  sessions)  and  McGuire  AFB  (used  in  the  data  collection  sessions).  Both  airfields 
were  constructed  from  the  published  approach  plates  (Department  of  Defense  (DoD)  Flight 
Information  Publication  (Terminal))  and  included  the  runways,  taxi  ways,  and  lighting 
systems,  as  well  as  the  control  tower,  hangers,  and  other  structures  at  the  field. 
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3.1  Visual  Meteorological  Conditions 


The  depiction  of  Visual  Meteorological  Conditions  (VMC)  consisted  of  a  simulated  full 
color  visual  scene  with  unlimited  visibility.  VMC  was  included  for  baseline  comparisons  (see 
Figure  6). 


Figure  6.  The  VMC  Sensor/Visibility  Condition  (VMC) 
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3.2  Instrument  Meteorological  Conditions  with  FLIR  Sensor 

The  computer-generated  scene  in  the  IMC  with  FLIR  Sensor  Condition  resembled  the 
image  produced  by  a  FLIR  sensor  operating  in  IMC.  This  condition  is  referred  to  as  the  FLIR 
Condition  and  is  illustrated  in  Figure  7.  It  provided  a  35°  x  25°  field-of-view  (FOV)  image  of 
the  airfield  environment  that  included  the  various  aspects  of  the  visual  scene  found  in  VMC 
(surface  texture,  runway  lights,  and  surrounding  structures).  The  simulated  FLIR  sensor 
image  gradually  improved  as  the  aircraft  proceeded  along  the  precision  approach,  in  such  a 
way  that  at  the  Category  (CAT)  I  decision  height  (200  ft  above  ground  level  (AGL)),  the 
sensor  image  displayed  the  runway  environment  with  an  1800  ft  forward  visibility.  The  FLIR 
sensor  image  was  displayed  throughout  the  entire  approach,  including  touchdown  and  rollout. 


Figure  7.  The  IMC  with  FLIR  Sensor/Visibility  Condition  (FLIR) 
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3.3  Instrument  Meteorological  Conditions  with  Millimeter  Wave  Radar 

The  OTW  view  during  the  IMC  with  MMWR  Condition  was  a  low-fidelity 
representation  of  MMWR  returns  from  a  series  of  comer  reflectors  positioned  along  the 
mnway  edgelines  and  across  the  mnway  threshold.  It  is  therefore  referred  to  as  the  Comer 
Reflectors  Condition.  The  image  of  the  mnway  outline  produced  by  the  MMWR  returns  was 
discernible  at  a  distance  of  approximately  1.5  miles  from  the  touchdown  zone  (see  Figure  8). 


Figure  8.  The  MMWR  Sensor/Visibility  Condition  (Comer  Reflectors) 
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3.4  No  Sensor 


The  No  Sensor  Condition  was  representative  of  an  IMC  visibility  condition  with  CAT  III 
weather  minimums  (0  ft  decision  height  and  700  ft  runway  visibility  range  (RVR)).  This 
condition  was  simulated  by  incorporating  a  dense  “fog”  that  obscured  the  OTW  scene  to  the 
desired  level  (see  Figure  9). 


Figure  9.  The  No  Sensor  Condition 
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4.  METHODOLOGY 


The  Autonomous  Landing  Guidance  (ALG)  study  incorporated  a  medium-fidelity 
simulation  to  address  the  study  objectives  that  were  introduced  in  Section  1 .  Twelve  pilots 
performed  two  experimental  tasks  in  the  TRAC  simulator:  a  Precision  Approach  Task  and  a 
Head-Up  Display  (HUD)  Assembly  Task.  Each  pilot  participated  in  a  series  of  training  and 
data  collection  sessions  for  the  Precision  Approach  Task  and  data  collection  sessions  for  the 
HUD  Assembly  Task.  Once  the  data  collection  sessions  were  finished,  each  pilot  completed  a 
Subjective  WORkload  Dominance  (SWORD)  form  and  a  set  of  questionnaires  that  are 
described  in  the  following  subsections.  The  use  of  simulation  and  the  general  conduct  of  the 
training  and  data  collection  sessions  are  also  described  below. 

4.1  Background  and  Experience  of  ALG  Pilots 

A  total  of  12  male  pilots  participated  in  the  ALG  study  over  a  period  of  seven  weeks.  All 
but  one  of  the  pilots  were  currently  active  and  most  had  experience  with  low-visibility 
approaches  and  landings.  Nine  of  the  pilots  had  Air  Force  transport  aircraft  experience.  Five 
were  Air  Force  or  Air  National  Guard  fighter  pilots  with  HUD  experience.  Three  pilots 
carried  commercial  ratings  and  were  actively  flying  for  a  passenger  airline.  The  background 
and  experience  of  the  pilots  are  summarized  in  Table  2.  Abbreviations  used  in  the  table  are  as 
follows:  AC  -  aircraft  commander,  IP  -  instructor  pilot,  MP  -  mission  pilot,  and  EP  -  evaluator 
pilot. 

4.2  The  Transport  Aircraft  Cockpit  Simulator 

The  ALG  HUD  study  was  conducted  in  the  Transport  Aircraft  Cockpit  (TRAC) 
Simulator,  which  is  a  reconfigurable  full  mission  simulator  with  dynamic  displays  and  full 
flight  control  capability.  Figure  10  provides  a  representation  of  the  TRAC  Simulator.  The 
simulator  cockpit  approximates  a  C-141  geometry  and  contains  pilot  and  co-pilot  stations  as 
well  as  an  experimenter’s  console  that  is  located  behind  the  co-pilot’s  station.  Control  yokes 
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are  provided  at  both  pilot  stations  with  the  throttle  quadrant  mounted  on  the  center  pedestal. 
Flight  instruments,  engine  instruments,  and  other  displays  are  graphically  drawn  on  three 
large-screen  cathode  ray  tubes  (CRTs)  mounted  in  the  instrument  panel.  Other  aircraft  system 
controls  and  displays  are  installed  or  mocked-up  as  required  for  studies.  Two  wide-angle 
collimated  (WAC)  display  windows  are  mounted  directly  in  firont  of  the  pilot  and  co-pilot 
stations. 
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Table  2.  Summary  of  the  Background  and  Experience  of  the  ALG  Pilots 


Subject 

No. 

Air  Force 
Transport 
Rating 

Air  Force 
Fighter 
Rating 

Commercial 

Rating 

Total 

Hours 

Low-Visibility 

Approaches 

HUD 

Sensors 

1 

AC 

3250 

2 

AC 

IP 

2300 

3 

IP 

3300 

4 

Flight  Lead 

Captain 

7500 

5 

IP 

2895 

6 

AC 

2940 

7 

MP 

705 

8 

EP 

2950 

9 

AC 

First  Officer 

2150 

10 

EP 

2150 

11 

EP 

7500 

12 

AC 

EP 

Captain 

5920 

In  the  conduct  of  the  ALG  study,  the  pilots  flew  from  the  pilot’s  flight  station  while  the 
co-pilot’s  station  remained  unattended.  The  test  engineer  sat  at  the  experimenter’s  console 
behind  the  co-pilot’s  station.  The  HUD  symbology  and  the  out-the-window  (OTW)  visual 
scene  (one  of  the  four  SensorA^isibility  Conditions)  were  presented  on  both  WAC  windows. 
Graphically  drawn  engine  instruments  were  provided  on  the  center  CRT  mounted  in  the 
instrument  panel.  No  other  head-down  flight  instruments  were  provided. 

4.3  Training 

Each  pilot  received  a  classroom  briefing  regarding  the  ALG  concept,  the  TRAC 
simulator,  and  the  HUD  symbology  dynamics  and  mechanization.  The  pilot  then  participated 
in  a  series  of  simulator  familiarization  flights,  which  included  practice  approaches  with  each 
of  the  16  combinations  of  the  HUD  Symbology  Sets  and  SensorA^isibility  Conditions.  The 
practice  approaches  were  flown  using  a  database  fashioned  after  the  Pope  Air  Force  Base 
(AFB)  airfield. 
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4.4  The  Precision  Approach  Task 


In  the  Precision  Approach  Task,  each  pilot  flew  two  replications  with  each  combination 
of  the  four  HUD  Symbology  Sets  (Basic/RWY,  Basic/RAW,  Basic/BOTH,  and  Full  HUD) 
and  the  four  SensorA^isibility  Conditions  (VMC,  FLIR,  Comer  Reflectors,  and  No  Sensor), 
resulting  in  a  total  of  32  precision  approaches.  The  approaches  were  initiated  five  miles  from 
the  touchdown  zone,  with  the  aircraft  configured  for  final  approach  (gear  down,  flaps  set)  and 
established  on  glideslope  and  on  localizer  course.  The  pilot  set  the  engine  throttles  to 
maintain  the  commanded  approach  speed  and  sink  rate  and  manually  flew  the  entire  approach 
to  touchdown.  A  variable  quartering  headwind  was  present  to  elicit  active  pilot  control  and  to 
exercise  the  HUD  symbology. 

The  following  objective  performance  measures  were  collected  at  a  rate  of  30  Hz  during 
the  Precision  Approach  Task:  Root-Mean-Square  (RMS)  glideslope  deviation,  RMS  localizer 
deviation,  RMS  airspeed  deviation  from  commanded  airspeed,  and  RMS  vertical  velocity 
deviation.  As  illustrated  in  Figure  11,  the  variables  were  collected  for  two  segments  of  the 
approach  -  “long  final”  and  “short  final.”  In  the  long  final  segment,  the  runway  environment 
was  not  visible  and  the  pilot  used  the  symbology  as  the  primary  flight  reference.  In  the  short 
final  segment,  the  runway  environment  was  either  visible  or  provided  via  sensor  imagery  with 
the  pilot  having  transitioned  to  the  “visual”  portion  of  the  approach.  The  two  segments  served 
to  identify  the  relative  contribution  of  sensor  imagery  and  symbology  to  landing  performance 
and  represent  the  Approach  Segment  Condition  in  later  sections  of  this  report.  The  discrete 
values  of  touchdown  deviation  from  the  centerline  and  touchdown  deviation  from  the  runway 
point  of  intercept  (RPI)  established  1000  ft  from  the  threshold  were  also  collected  for  each 
approach  and  analyzed  as  objective  performance  measures. 
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. 

’’Short  final"  data  Touchdown 

collection  window 

Figure  11.  The  Two  Data  Collection  Windows  of  the  Precision  Approach  Task 

4.5  The  HUD  Assembly  Task 

Upon  completing  the  Precision  Approach  Task,  each  pilot  performed  the  HUD 
Assembly  Task.  In  this  task,  each  pilot  viewed  a  variety  of  static  Approach  Conditions 
while  occupying  the  pilot  station  in  the  TRAC  simulator.  Each  pilot  was  presented  with  a 
total  of  nine  Approach  Conditions,  each  representative  of  the  situations  they  experienced 
in  the  Precision  Approach  Task.  The  nine  Approach  Conditions  are  shown  in  Table  3. 

Each  condition  began  with  the  Full  HUD  Symbology  Set  displayed.  The  pilot  then 
verbally  selected  the  ERJD  symbols  he  wanted  to  retain  or  delete.  The  test  engineer 
selected  or  deselected  the  symbols  using  the  experimenter’s  console,  and  the  resultant 
HUD  symbology  set  was  displayed  in  real-time.  Once  the  pilot  was  satisfied  with  his 
selections  for  a  given  condition,  the  selections  were  saved  to  a  data  file.  Two  HUD 
symbology  sets  were  constructed:  an  absolute  minimum  symbology  set  and  an  optimum 
symbology  set.  For  the  minimum  symbology  set,  the  pilot  was  asked  to  identify  the 
minimum  symbology  he  would  require  to  safely  complete  an  approach.  For  the  optimum 
symbology  set,  the  pilot  was  asked  to  identify  the  symbology  set  he  believed  would  result 
in  the  best  performance. 


"Long  final"  data 
collection  window 
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Table  3.  The  Approach  Conditions  for  the  HUD  Assembly  Task 


Condition 

SensorA/isibility 

A/C  Orientation 

A/C  Position 

1 

VMC 

no  crab 

long  final 

2 

FUR 

crab 

long  final 

3 

FUR 

no  crab 

long  final 

4 

FUR 

crab 

short  final 

5 

FUR 

no  crab 

short  final 

6 

Corner  Reflectors 

crab 

long  final 

7 

Corner  Reflectors 

no  crab 

long  final 

8 

Corner  Reflectors 

crab 

short  final 

9 

Corner  Reflectors 

no  crab 

short  final 

4.6  Questionnaires 

After  completing  the  HUD  Assembly  Task,  each  pilot  provided  a  subjective  workload 
evaluation  as  well  as  ratings  and  comments  collected  using  a  questionnaire.  The  questionnaire 
is  provided  in  Appendix  A.  The  workload  evaluation  was  accomplished  using  the  SWORD 
technique  (Vidulich,  1989).  A  SWORD  form  was  used  to  collect  pairwise  comparisons  of 
workload  across  the  HUD  Symbology  Sets.  The  comparisons  were  then  entered  into  the 
SWORD  computer  program  that  calculated  workload  ratings  for  the  HUD  Symbology  Sets. 
The  questionnaire  provided  acceptability  ratings  regarding  all  aspects  of  the  various  HUD 
Symbology  Sets,  as  well  as  comments  that  augmented  the  acceptability  ratings. 

4.7  Experimental  Design 

In  the  Precision  Approach  Task,  each  of  the  four  HUD  Symbology  Sets  were  flown  in 
each  of  the  SensorWisibility  Conditions,  resulting  in  16  experimental  conditions.  A  repeated 
measures  design  was  employed,  in  which  two  replications  of  each  of  the  16  experimental 
conditions  were  flown.  The  presentation  order  of  the  experimental  conditions  was  blocked 
with  respect  to  the  SensorWisibility  Condition,  so  that  each  of  the  four  HUD  Symbology  Sets 
were  flown  in  a  specific  SensorWisibility  Condition  before  proceeding  to  the  next 
SensorWisibility  Condition.  The  presentation  order  of  the  blocked  SensorWisibility 
Conditions  was  counterbalanced  across  pilots  using  the  Latin  Squares  technique  to  reduce  the 
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potential  for  order  effects.  Within  each  SensorWisibility  block,  the  presentation  order  of  the 
HUD  Symbology  Sets  was  counterbalanced  across  pilots,  further  reducing  the  potential  for 
order  effects. 

For  the  HUD  Assembly  Task,  each  pilot  received  the  nine  Approach  Conditions  in  the 
order  shown  in  Table  3.  In  each  case,  the  “safe  minimum”  symbology  set  was  identified  first, 
followed  by  the  “optimum”  symbology  set. 
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5.  RESULTS  AND  DISCUSSION 


This  section  provides  a  summary  of  the  data  analysis  methods,  significant  Analysis  of 
Variance  (ANOVA)  results,  findings  from  the  questionnaire  data,  and  a  discussion  of  the 
cumulative  results  for  each  analysis  objective.  The  analysis  objectives  paralleled  the  study 
objectives:  1)  Determine  the  relative  effects  of  different  symbology  sets  and  their  inherent 
levels  of  clutter  on  pilot  performance,  pilot  acceptance,  and  pilot  workload;  2)  Determine  the 
effect  of  sensor  imagery  on  pilot  performance,  pilot  acceptance,  and  pilot  workload  in  an 
Autonomous  Landing  Guidance  (ALG)  context;  and  3)  Identify  “safe  minimum”  and 
“optimum”  symbology  sets. 

5.1  Objective  1 

The  first  objective  of  the  ALG  study  was  to  determine  the  relative  effects  of  different 
symbology  sets  and  their  inherent  levels  of  clutter  on  pilot  performance,  pilot  acceptance,  and 
pilot  workload. 

5.1.1  Data  Analysis 

The  analysis  for  Objective  1  was  conducted  by  assessing  pilot  performance  data, 
relative  workload  ratings,  and  questionnaire  responses.  Statistical  comparisons  of  the 
performance  levels  and  workload  ratings,  using  ANOVA  techniques,  were  performed  across 
the  different  head-up  display  (HUD)  Symbology  Sets  and  within  the  SensorA/isibility 
Conditions,  as  well  as  the  Approach  Segment  Conditions.  Post-hoc  ANOVAs  and  Duncan 
tests  were  performed  on  the  main  and  interaction  effects  to  identify  the  nature  of  the  statistical 
differences.  Appendix  B  provides  the  F-statistics,  significance  levels,  and  means  associated 
with  the  tested  effects.  For  the  questionnaire  ratings,  experimenters  computed  frequencies  and 
averages  and  summarized  pilot  comments.  The  ratings  and  comments  supplied  by  the  pilots 
are  summarized  in  Appendix  C. 
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Table  4  provides  a  summary  of  the  tested  effects  that  were  relevant  to  this  objective  and 
their  results.  Cells  with  an  a  <  .05  indicate  a  statistically  significant  finding.  The 
abbreviations  used  in  the  table  are  as  follows;  HUD  -  HUD  Symbology  Set;  W  -  vertical 
velocity;  IAS  -  indicated  airspeed;  TD  -  touchdown  zone;  and  CENTER  -  runway  centerline. 


Table  4.  Analyses  Performed  for  Objective  1 


MAIN  EFFECTS 

1 

j  '  '  • 

1  '  ' 

[ 

1' 

HUD 

a  <.05 

a  <  .05 

a  <  .05 

HI 

1  a<.05 

HI 

IHI 

INTERACTION  EFFECTS 

. .  > 

f  • 

i:.;  1 

HUD  X  Api^oach  Segment 

a  <.05 

a  <  .05 

a  <.05 

IHI 

HUD  X  SensorA^isibility 

a  <.05 

a  <  .05 

a  <.05 

a  <  .05 

^■1 

\^m 

HUD  X  Segment  X  SensOT/Visibility 

a  <.05 

|H 

I^H 

5.1.2  ANOVA  Results 

The  following  results  are  based  on  the  ANOVA  procedures  and  post-hoc  tests  that  were 
applied  to  the  performance  and  workload  data.  Basic/RWY,  Basic/RAW,  Basic/BOTH,  and 
Full  HUD  refer  to  the  HUD  Symbology  Sets  described  in  Section  2. 

•  In  all  of  the  of  SensorA^isibility  Conditions  and  in  both  Approach  Segments,  the 
pilots  maintained  glideslope  and  localizer  most  precisely  with  the  Basic/RAW  and 
the  Basic/BOTH  Symbology  Sets,  and  least  precisely  with  the  Basic/RWY  set  (see 
Figures  12  and  13).  An  analysis  of  the  triple  interaction  involving  Approach 
Segment  (Figure  12)  revealed  a  similar  pattern  for  the  long  and  short  final 
approach  segments,  which  differed  in  magnitude  only  (see  Appendix  B).  Also, 
when  using  the  Full  HUD,  pilots  were  less  precise  at  staying  on  the  localizer  (see 
Figure  13). 
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Mean  RMS  Localizer  Deviation  (degrees)  Mean  RMS  Glideslope  Deviation  (degrees) 
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Igure  12.  Glideslope  Deviation  Performance  as  a  Function  of  Sensor/Visibility 
Condition  and  HUD  Symbology  Set 
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Figure  13.  Mean  RMS  Localizer  Deviation  as  a  Function  of  HUD  Symbology  Set 
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•  In  all  SensorA^isibility  Conditions  except  VMC,  pilots  most  accurately  maintained 
the  desired  vertical  velocity  when  using  the  Basic/RWY  HUD  set. 

•  Pilots  rated  workload  higher  for  the  Basic/RWY  and  Full  HUD  sets  than  for  the 
Basic/RAW  and  Basic/BOTH  HUD  Symbology  Sets  (see  Figure  14). 
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Figure  14.  Mean  Workload  Rating  Across  Subjects  as  a  Function  of  HUD  Symbology 

Set 


5.1.3  Questionnaire  Results 

The  following  results  are  based  on  the  pilot  comments  and  frequency  distributions  and 
averages  calculated  for  the  questionnaire  rating  scales.  When  asked  to  rate  their  ability  to 
accomplish  approach  tasks  (e.g.,  control  lateral  deviation,  control  vertical  deviation,  control 
aircraft  performance,  determine  desired  touch  down  point,  and  safely  fly  the  approach)  the 
pilots  most  often  rated  the  Basic/RAW  and  Basic/BOTH  formats  higher  than  the  Basic/RWY 
and  Full  HUD  formats  (see  Figures  15,  16,  and  17).  For  each  figure,  the  rating  scale  is  as 
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follows:  5  -  Very  Acceptable;  4  -  Moderately  Acceptable;  3  -  Borderline;  2  -  Moderately 
Unacceptable;  1  -  Very  Unacceptable.  The  prevailing  pattern  for  each  figure  is  that  the 
Basic/RAW  and  Basic/BOTH  HUD  Symbology  Sets  received  higher  acceptability  ratings 
than  the  Basic/RWY  and  Full  HUDs. 
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Figure  15.  The  Average  Rating  for  the  Ability  to  Monitor  and  Control  Lateral 
Deviation  as  a  Function  of  Sensor/Visibility  Condition  and  HUD  Symbology  Set 
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SensorA/isibility  Condition 


Figure  1 6.  The  Average  Rating  for  the  Ability  to  Monitor  and  Control  Vertical 
Deviation  as  a  Function  of  Sensor/Visibility  Condition  and  HUD  Symbology  Set 
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Figure  1 7.  The  Average  Rating  for  the  Ability  to  Safely  Fly  the  Approach  as  a 
Function  of  Sensor/Visibility  Condition  and  HUD  Symbology  Set 


Figure  18  illustrates  that  the  pilots  reported  more  safety  issues  associated  with  the 
Basic/RWY  HUD  and  Full  HUD  than  for  the  Basic/RAW  HUD  and  Basic/BOTH  HUD.  The 
figure  also  shows  that  the  majority  of  pilots  expressed  safety  concerns  with  using  the 
Basic/RWY  HUD  for  low-visibility  approaches  because  it  did  not  provide  sufficient  approach 
guidance  information.  Specifically,  pilots  indicated  that  raw  vertical  and  lateral  deviation 
data,  a  course  indication,  distance  measuring  equipment  (DME),  and  surface  wind  direction 
and  magnitude  were  needed  as  part  of  the  Basic/RWY  set.  Pilot  recommendations  were 
particularly  salient  for  the  long  Approach  Segment  where  a  sensor  or  visual  image  was  not 
available. 
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Sensor/Visibility  Condition 

Figure  18.  The  Number  of  Pilots  that  Responded  “Yes  ”  to  the  Question  of  Safety 
Issues  as  a  Function  of  HUD  Symbology  Set  and  Sensor  Availability 

Also,  half  of  the  pilots  identified  safety  concerns  with  the  Full  HUD  set  (see  Figure  18). 
Safety  issues  resulted  from  the  HUD  format  containing  too  much  information  and  being  too 
cluttered.  Pilots  commented  that  the  high  information  density  and  resultant  clutter  was 
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distracting  and  obscured  the  visual  scene  and  important  symbology.  Comments  indicated  that 
the  flare  and  ground  annunciators,  as  well  as  the  rising  runway,  speed  deviation  cue, 
acceleration  cue,  and  artificial  runways  were  largely  unnecessary.  Conversely,  the  pilots 
expressed  very  few  safety  concerns  with  the  Basic  RAW  and  Basic/BOTH  sets. 

The  HUD  Symbology  Sets  suffered  clutter  problems  in  varying  degrees  (see  Figure  19). 
The  Basic/RWY  and  the  Basic/RAW  HUDs  had  the  least  clutter  problems,  while  the  Full 
HUD  suffered  the  most  because  of  clutter.  The  pilots  consistently  commented  that  the  aircraft 
reference  symbol  (part  of  the  Basic  HUD  set)  was  too  large  and  provided  information  of 
dubious  utility. 


SensorA/isibility  Condition 


Figure  19.  The  Number  of  Pilots  that  Responded  “Yes  ”  to  the  Question  of  Clutter 
Issues  as  a  Function  of  HUD  Symbology  Set  and  Sensor  Availability 


5.1.4  Objective  1  Discussion 

The  performance  and  workload  results  show  a  consistent  pattern  as  a  function  of  the  HUD 
Symbology  Sets.  The  pilots  performed  better  and  rated  workload  lower  with  both  the 
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Basic/RAW  and  Basic/BOTH  Symbology  Sets  than  with  the  Basic/RWY  and  Full  HUD 
Symbology  Sets.  Responses  to  questionnaire  items  showed  the  same  pattern,  indicating  the 
Basic/RWY  format  did  not  provide  adequate  information,  and  the  Full  HUD  format  contained 
too  much  information. 

The  findings  suggest  that  a  continuum  of  symbology  density  (and  resultant  clutter)  exists, 
with  a  minimum  symbology  density  (or  information  density)  at  one  end  and  a  maximum 
symbology  density  at  the  other.  As  expected,  the  Precision  Approach  Task  performance  data 
suggest  that  a  minimum  amoimt  of  symbology  (information)  must  be  provided  to  obtain 
acceptable  levels  of  performance.  Adding  symbology  (information)  can  improve  performance 
until  a  point  on  the  continuum  is  reached;  thereafter,  excessive  clutter  begins  to  degrade 
performance.  In  reference  to  the  performance  data  from  the  Precision  Approach  Task,  and  the 
workload  and  questionnaire  data,  the  minimum  symbology  provision  should  include  more 
information  than  is  available  with  the  Basic/RWY  HUD  set,  and  less  than  is  available  with  the 
Full  HUD. 

5.2  Objective  2 

The  second  objective  of  the  ALG  study  was  to  determine  the  effect  of  sensor  imagery  on 
pilot  performance,  pilot  acceptance,  and  pilot  workload  in  an  ALG  context. 

5.2.1  Data  Analysis 

As  with  Objective  1,  the  analyses  for  the  Objective  2  involved  the  assessment  of  the 
pilot  performance  data,  relative  workload  ratings,  and  questioimaire  responses.  The  Precision 
Approach  performance  data  and  the  workload  ratings  were  investigated  using  ANOVA 
techniques  as  a  function  of  the  HUD  Symbology  Sets  and  within  the  Sensor/Visibility 
Conditions.  Post-hoc  ANOVAs  and  Duncan  tests  were  performed  on  the  main  and  interaction 
effects  to  identify  the  nature  of  the  statistical  differences.  Appendix  B  provides  the  F- 
statistics,  significance  levels  and  means  associated  with  all  of  the  tested  effects.  Frequencies 
and  averages  were  computed  for  the  questionnaire  ratings  and  the  pilot  comments  were 
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summarized.  Table  5  provides  a  summary  of  the  tested  effects  that  were  relevant  to  this 
objective  and  their  results.  Cells  with  an  a  <  .05  indicate  a  statistically  significant  result.  The 
abbreviations  used  in  the  table  are  as  follows;  HUD  -  HUD  Symbology  Set;  W  -  vertical 
velocity;  IAS  -  indicated  airspeed;  TD  -  touchdown  zone;  and  CENTER  -  runway  centerline. 

Table  5.  Results  of  the  ANOVAs  Accomplished for  Each  Performance  Variable  as 
a  Function  of  Sensor/Visibility  Condition 


MAIN  EFFECTS 

Sensor/Visibility 

INTERACTION  EFFECTS 
HUD  X  SensorA^isibility 


5.2.2  ANOVA  Results 

The  following  results  are  based  on  the  ANOVA  procedures  and  post-hoc  tests  that  were 
applied  to  the  performance  data.  A  detailed  treatment  of  the  main  and  interaction  effects  may 
be  found  in  Appendix  B.  The  results  of  the  workload  analysis  are  not  reported  because  no 
significant  differences  were  found  as  a  function  of  SensorA^isibility  Conditions: 

•  Pilots  touched-down  closer  to  the  runway  centerline  in  the  VMC  and  FLIR 
Conditions  than  in  the  Comer  Reflector  and  No  Sensor  Conditions  (see  Figure  20). 

•  The  interactions  between  HUD  Symbology  Set  and  SensorA^isibility  Condition 
were  examined,  but  produced  only  one  interesting  finding.  With  the  Full  UDD 
Symbology  Set,  the  pilots  flew  glideslope  most  accurately  in  the  VMC  Condition 
and  least  accurately  in  the  FLIR  and  No  Sensor  Conditions. 
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With  the  Basic/RAW  Symbology  Set,  the  pilots  maintained  airspeed  and  vertical 
velocity  most  precisely  in  the  VMC  Condition  and  least  precisely  with  the  No 
Sensor  Condition. 
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Figure  20.  Mean  Touchdown  Deviation  from  Centerline  as  a  Function  of 

Sensor/Visibility  Condition 

5,2.3  Questionnaire  Results 

Pilots  consistently  rated  their  ability  to  accomplish  the  approach  tasks  as  between 
''Moderately  Acceptable”  and  “Borderline”  when  flying  in  the  No  Sensor  Condition  (see 
Figures  15,  16,  and  17).  This  trend  may  have  been  influenced  by  the  low  contrast  between  the 
symbology  and  the  dense  white  “fog”  background  in  the  No  Sensor  Condition.  The  pilots  also 
consistently  rated  their  ability  to  safely  perform  the  approach  (and  specific  approach  tasks)  as 
very  low  (e.g.,  below  moderately  acceptable)  when  using  the  Basic/RWY  HUD  before  the 
sensor  image  became  visible.  Average  pilot  ratings  on  the  ability  to  accomplish  approach 
tasks  did  not  appear  to  depend  on  the  SensorA/isibility  Conditions  for  the  other  three  HUD 
Symbology  Sets.  Overall,  pilots  were  very  positive  about  the  benefits  of  the  sensor  imagery 
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for  low- visibility  approaches.  When  asked  about  the  viability  of  the  ALG  concept,  the  pilots 
expressed  high  confidence  in  both  the  concept  and  the  resulting  integration  of  symbology  and 
sensor  imagery  on  a  HUD  display. 

5.2.4  Objective  2  Discussion 

As  expected,  both  performance  and  subjective  data  showed  performance  benefits  as  a 
result  of  adding  sensor  imagery  of  the  runway  environment.  However,  the  SensorWisibility 
Conditions  did  not  affect  pilot  workload  ratings.  The  pilots  performed  the  same  in  the  FLIR 
and  VMC  Conditions.  The  subjective  data  also  indicated  that  the  perceived  benefits  of  the 
sensor  imagery  were  greater  when  the  HUD  symbology  provided  limited  information.  The 
relatively  low-quality  sensor  image  provided  in  the  Comer  Reflector  Condition  did  not 
improve  performance  over  the  No  Sensor  Condition.  This  may  have  resulted  from  the  low- 
fidelity  representation  of  MMWR  or  from  the  absence  of  visual  cues  such  as  texture,  surface 
features,  or  ground  objects.  However,  these  findings  emphasize  the  importance  of  a  high 
quality  sensor  imagery  in  ALG  applications. 

5.3  Objectives 

The  third  objective  of  the  ALG  study  was  to  identify  “safe  minimum”  and  “optimum” 
ALG  HUD  symbology  sets. 

5.3.1  Data  Analysis 

Objective  3  was  directly  addressed  by  evaluating  the  HUD  Assembly  Task  data. 
However,  much  of  the  pilot  performance  and  questionnaire  data  addressing  clutter  is  also 
relevant  to  this  objective.  A  frequency  analysis  was  performed  on  the  HUD  Assembly  Task 
selections  to  identify  how  many  pilots  selected  each  possible  symbol  for  each  Approach 
Condition. 
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5.3.2  Frequency  Analysis  Results 


Table  6  shows  the  results  of  the  frequency  analysis  performed  on  the  HUD  Assembly 
Task  data.  Safe  minimum  and  optimum  selections  are  shown  in  columns  for  each  Approach 
Condition.  Each  row  represents  a  HUD  symbol.  An  “X”  indicates  that  at  least  half  of  the 
pilots  selected  the  symbol  for  the  Approach  Condition  represented  in  that  column.  In  the  far 
left  column,  a  “B”  indicates  the  symbol  was  provided  as  part  of  the  Basic  HUD  symbol  set,  an 
“A”  indicates  approach  symbology,  and  an  “F”  indicates  the  symbols  were  included  in  the 
Full  HUD.  On  the  top  row,  CR  indicates  Comer  Reflectors. 

The  following  observations  are  based  on  an  inspection  of  the  frequency  analysis  results: 

•  The  minimum  or  optimum  symbology  sets  did  not  appear  to  depend  on  the  type  of 
sensor  provided  or  whether  a  crab  condition  existed. 

•  The  minimum  and  optimum  symbology  sets  did  appear  to  differ  between  short  and 
long  final  conditions. 

•  The  pilots  selected  most  of  the  Basic  HUD  symbology  (except  the  digital 
groundspeed  symbol,  the  aircraft  reference  symbol,  the  digital  radar  altimeter 
provision,  and  the  roll  limit  symbology)  for  display  in  all  of  the  Approach 
Conditions. 

•  The  majority  of  pilots  preferred  the  raw  localizer  and  glideslope  deviation 
symbology  over  the  artificial  runway  symbol  for  executing  the  approaches. 

Two  of  the  symbols  from  the  Full  HUD  set  (digital  DME  and  the  surface  wind  vector) 
were  selected  for  inclusion  in  the  preferred  sets.  The  pilots  also  selected  two  mode-dependent 
symbologies  for  long  final  conditions:  the  course  deviation  indicator  (CDI)  needle  and  the  to- 
from  indicator. 
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Table  7  summarizes  the  pilots’  selections  for  minimum  and  optimal  symbology  sets. 
Based  on  the  frequency  analysis  that  was  performed  on  the  HUD  Assembly  Task  data  (see 
Table  6),  separate  symbology  sets  are  recommended  for  the  short  and  long  final  approach 
segments.  Note  also  that  nearly  all  of  the  symbols  were  retained  throughout  the  entire 
approach,  suggesting  that  the  sensor  image  was  used  as  supplementary  information,  rather 
than  as  a  replacement  for  selected  symbology.  The  retention  of  symbology  is  due  in  part  to 
the  range  limitations  of  the  sensors  modeled  in  this  study  (and  sensors  in  general).  The 
recommended  minimum  and  optimum  symbology  sets  for  each  Approach  Segment  are 
represented  in  Figures  21, 22, 23,  and  24. 


Table  7.  HUD  Symbology  Recommendations  as  a  Function  of  Approach  Segment 


Long  Final 

Short  Final 

Minimum  Symbol  Set 

Basic  +  Raw 

minus  Aircraft  Reference 
Digital  Groundspeed 
plus  Selected  Course 

DME 

“GDI  needle” 

Basic  +  Raw 

minus  Aircraft  Reference 
Digital  Groundspeed 
plus  Selected  Course 

DME 

Optimum  Symbol  Set 

Basic  +  Raw 

minus  Aircraft  Reference 
Digital  Groundspeed 
plus  Selected  Course 

DME 

“CDI  Needle” 

Wind  Vector 

Flight  Path  Guidance 
Cue 

Basic  +  Raw 

minus  Aircraft  Reference 
Digital  Groundspeed 
plus  Selected  Course 

DME 

Wind  Vector 

Flight  Path  Guidance 
Cue 

Artificial  Runway 
Rising  Runway 
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Figure  21.  The  Recommended  Minimum  Symbology  for  Short  Final 


Figure  22.  The  Recommended  Maximum  Symbology  for  Short  Final 
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Figure  23.  The  Recommended  Minimum  Symbology  Set  for  Long  Final 


Figure  24.  The  Recommened  Maximum  Symbology  Set  for  Long  Final 
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5.3.3  Objective  3  Discussion 


The  performance,  workload,  and  questionnaire  results  along  with  the  frequency  analysis 
results  provide  converging  evidence  that  an  optimum  HUD  symbology  set  for  ALG 
applications  contains  more  symbology  than  is  provided  in  the  Basic/RWY  Symbology  Set  and 
less  symbology  than  is  found  in  the  Full  HUD.  The  performance  data  identifies  the 
Basic/RWY  HUD  as  possibly  being  below  the  “safe  minimum”  and  certainly  not  “optimum” 
from  a  performance  standpoint.  The  clutter  problems  identified  by  the  pilots  would  also 
eliminate  the  Full  HUD  from  consideration  as  the  “optimum”  symbology  set.  Due  to  the 
excessive  workload  and  the  associated  questionnaire  comments,  this  research  strongly 
suggests  that  the  artificial  nmway  is  inadequate  as  a  guidance  symbology  and  should  not  be 
considered  for  either  the  “minimum  safe”  or  “optimum”  symbology  set.  Conversely,  because 
the  raw  data  tended  to  reduce  workload,  both  the  raw  localizer  deviation  and  the  raw 
glideslope  deviation  should  be  strongly  considered  for  inclusion. 

5.4  General  Discussion 

Recall  that  the  current  study  is  intended  to  lead  to  HUD  symbology  requirements  for  an 
ALG  system.  To  that  end,  the  results  of  the  current  study  provide  a  recommended  symbology 
set  for  ALG  applications  that  is  supported  by  pilot  performance  and  pilot  subjective  data. 
With  few  exceptions,  the  current  recommendations  for  approach  symbology  are  consistent 
with  previously  published  HUD  information  requirements  and  recommendations,  such  as 
MIL-STD-1787,  Weintraub  and  Ensing  (1992),  Ercoline  and  Weinstein  (1993),  and  Hughes, 
et  al.  (1992). 

The  single  inconsistency  between  the  results  of  this  study  and  most  HUDs  currently  in  the 
field  is  the  presentation  of  energy  management  cues  (longitudinal  acceleration  and  angle-of- 
attack  (AOA)/airspeed  deviation).  Less  than  half  of  the  pilots  participating  in  this  study 
selected  these  cues  for  inclusion  in  either  the  “minimum  safe”  or  “optimum”  sets,  yet  they  are 
provided  in  most  fielded  HUDs.  The  lack  of  pilot  familiarity  with  the  operation  of  these 
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symbols  may  accoimt  for  the  inconsistency.  Most  of  the  pilots  in  the  evaluation  were 
transport  pilots  with  little  or  no  HUD  experience,  and  therefore  had  no  previous  experience 
with  energy  management  cues.  Participation  in  the  study  may  not  have  provided  sufficient 
familiarity  with  the  cues  because  each  approach  flown  in  the  Precision  Approach  Task  began 
with  the  simulator  stabilized  on  localizer  and  glideslope  with  the  flaps  and  throttles  already 
correctly  set.  These  relatively  stable  conditions  did  not  heavily  exercise  the  energy 
management  symbology.  If  the  pilots  did  not  use  and  understand  the  symbols,  they  could  not 
recognize  their  value. 

Even  though  the  current  results  reflect  a  high  degree  of  consistency  with  previous  HUD 
development  work,  some  issues  should  be  considered  when  generalizing  these  results.  First, 
the  design  and  information  elements  provided  to  the  pilots  were  obviously  inseparable  and  the 
results  presented  are  based  entirely  on  the  Sextant  design.  In  some  cases,  the  design  or 
dynamics  of  a  symbol  (e.g.,  size  of  the  aircraft  reference  symbol)  appeared  to  have  influenced 
the  study  results,  such  as  by  unnecessarily  cluttering  the  display  or  causing  confusion. 
Therefore,  a  different  symbology  set  could  have  produced  slightly  different  results.  Second, 
the  low  fidelity  of  the  sensor  should  also  be  considered  when  generalizing  results. 
Specifically,  the  Comer  Reflector  Condition  was  an  oversimplified  representation  of  MMWR 
sensor  imagery.  In  the  study,  the  comer  reflectors  were  shown  as  dots  of  white  over  a  black 
background.  The  scene  did  not  show  any  “noise”  or  returns  fi'om  other  ground  objects.  The 
use  of  the  information  provided  by  the  comer  reflectors  may  have  been  better  or  worse  in  a 
more  realistic  scene.  Finally,  as  in  any  simulation,  the  results  obtained  should  not  be  used  to 
predict  real-world  pilot  performance  or  workload.  However,  a  similar  pattern  of  results  would 
be  expected  for  real-world  conditions. 
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6.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  conclusions  drawn  from  this  evaluation  are  directly  related  to  the  original  objectives 
of  this  study:  1)  Determine  the  relative  effects  of  different  symbology  sets  and  their  inherent 
levels  of  clutter  on  pilot  performance;  2)  Determine  the  effect  of  sensor  imagery  on  pilot 
performance  in  an  Autonomous  Landing  Guidance  (ALG)  task;  and,  3)  Identify  “safe 
minimum”  and  “optimum”  symbology  sets. 

A  number  of  conclusions  can  be  drawn  regarding  the  comparison  of  the  different  head-up 
display  (HUD)  Symbology  Sets  and  their  inherent  levels  of  clutter.  First,  the  Basic 
Symbology  Set,  with  the  exception  of  the  aircraft  reference  symbol,  provides  a  good 
foundation  to  build  on.  Adding  the  raw  data  to  this  set  is  preferred  over  the  addition  of  the 
runway  symbol,  with  virtually  no  dividend  derived  from  adding  both.  Finally,  this  research 
determined  that  the  Full  HUD  set  provided  too  much  information,  at  least  for  those  pilots  with 
a  limited  amovmt  of  HUD  experience. 

In  terms  of  identifying  a  “safe  minimum”  symbology  set  and  an  “optimum”  symbology 
set,  this  research  indicated  that  these  sets  were  conditional;  that  is,  the  two  symbology  sets 
were  primarily  influenced  by  the  relative  distance  from  the  airstrip.  The  two  conditions  of 
distance  to  the  airstrip  investigated  have  been  referred  to  as  “long  final”  (distance  measuring 
equipment  (DME)  greater  than  approximately  1.5  Nm)  and  “short  final”  (DME  less  than 
approximately  1.5  Nm).  This  differentiation  is  a  direct  result  of  the  provision  of  the  sensor 
image,  which  can  begin  to  provide  information  at  approximately  1.5  DME.  Each  set  should 
include,  as  a  minimum,  the  Basic  Symbology  Set  minus  the  aircraft  reference  symbol  and  the 
digital  groundspeed.  The  symbols  added  to  the  Basic  Symbology  Set  to  arrive  at  the  “safe 
minimum”  and  “optimum”  sets  were  provided  in  Table  7  in  Section  5,  as  a  function  of 
Approach  Segment  (“long  final”  versus  “short  final”). 

Because  the  current  study  evaluated  a  single  symbology  mechanization  (Sextant’s 
symbology  design)  and  employed  low  fidelity  sensor  imagery,  the  researchers  would  be 
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presumptuous  to  present  these  recommendations  as  the  standard  for  all  ALG  HUDs.  Rather, 
these  recommendations  are  intended  to  serve  as  a  guide,  allowing  for  deviations  as  particular 
instances  warrant.  Recalling  the  continuum  of  symbology  availability,  with  no  symbology  at 
one  end  and  the  Full  HUD  Symbology  Set  at  the  other,  the  recommendations  found  in  Table  7 
in  Section  5,  represent  a  functional  set  of  symbols  for  use  in  a  small  variety  of  ALG  precision 
approach  conditions.  Additionally,  the  differentiation  between  the  “optimum”  and 
“minimum”  symbology  sets  could  serve  as  a  guide  for  a  declutter  application,  with  the 
“optimum”  set  being  provided  as  the  default  and  the  “minimum”  set  available  as  a  pilot- 
selectable  declutter  option. 

Concerning  the  effect  of  the  addition  of  sensor  imagery,  the  researchers  have  concluded 
from  the  analyses  that  the  addition  of  a  sensor  image  (like  a  forward-looking  infrared  (FLIR) 
sensor  image)  will  elicit  performance  similar  to  that  found  in  Visual  Meteorological 
Conditions  (VMC).  Unfortunately,  this  conclusion  could  not  be  extended  to  the  Comer 
Reflectors  Conditions.  The  researchers  believe  that  this  image  suffered  from  a  lack  of  fidelity, 
as  it  was  a  very  cmde  approximation  of  a  millimeter  wave  radar  (MMWR)  return  from  an 
airfield  equipped  with  comer  reflectors.  The  Comer  Reflectors  Conditions  did  not  include  any 
other  returns  from  the  airfield  environment  and  therefore  lacked  the  peripheral  cueing  that  was 
available  with  the  FLIR  sensor  image.  The  researchers  have  concluded,  however,  that  an 
acceptable  level  of  performance  could  be  accomplished  with  a  variety  of  sensor  images  (e.g., 
MMWR,  low-light  TV),  provided  they  are  of  nominal  quality  and  serve  to  extend  the  “visual” 
portion  of  an  ALG  approach. 

As  follow-up  to  this  study,  several  follow-on  activities  are  recommended.  The 
symbology  provisions  should  be  evaluated  using  higher  fidelity  sensor  models;  if  possible, 
using  actual  FLIR  or  MMWR  images  rather  than  simulated  versions  of  the  same.  Also,  we 
recommend  that  this  evaluation  be  performed  with  the  sensor  imagery  and  symbology 
presented  on  actual  HUD  hardware  mounted  in  the  simulator,  as  opposed  to  projecting  the 
HUD  symbology  on  a  single  display  medium  (the  wide-angle  collimated  (WAC)  vdndows) 
with  the  sensor/visibility  scene.  Performing  a  study  in  this  fashion  would  tend  to  eliminate 
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some  of  the  artificiality  and  limitations  associated  with  the  current  simulation.  Finally, 
suggestions  are  to  replicate  this  evaluation  with  other  HUD  symbology  sets,  most  notably  the 
MIL-STD  HUD.  This  replication  would  verify  that  the  information  necessary  in  an  ALG 
context  remains  the  same  regardless  of  the  particular  symbology  mechanization  incorporated. 
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APPENDIX  A  -  QUESTIONNAIRE 
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Introduction 


The  questionnaire  used  in  the  Head-Up  Display  (HUD)  Symbology  Evaluation  for  the 
Autonomous  Landing  Guidance  (ALG)  System  is  provided  in  this  Appendix.  A  summary  of  the 
pilots’  responses  is  provided  in  Appendix  C.  The  questionnaire  was  used  to  ellicit  the  following 
information  from  the  12  pilots  that  participated  in  the  study: 

♦  Biographical  data 

♦  An  evaluation  of  the  Basic  Plus  Runway  Symbology  Set 

♦  An  evaluation  of  the  Basic  Plus  Raw  Deviation  Data  Symbology  Set 

♦  An  evaluation  of  the  Basic  Plus  Both  Runway  and  Raw  Deviation  Data  Symbology  Set 

♦  An  evaluation  of  the  Full  Sextant  HUD 

♦  General  information  concerning  the  TRAC  simulator,  the  ALG  concept,  and  the  ALG 
study  as  conducted  by  this  laboratory 
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ALG  SYSTEMS  QUESTIONNAIRE 


The  following  questionnaire  will  investigate  the  utility  of  each  of  the  four  HUD  configurations 
presented  in  this  Autonomous  Landing  Guidance  (ALG)  study.  Each  display  will  be  treated 
separately.  Space  has  been  provided  for  additional  comments  after  many  of  the  questions.  If  you 
need  more  space,  please  continue  on  the  back  of  the  page.  Any  opinions  and  ideas  you  may  have 
regarding  the  improvement  of  the  displays’  operational  utility  and  reduction  of  crew  workload 
will  be  greatly  appreciated. 

Name _ 

Subject  Number _ 


BIOGRAPHICAL  DATA 


Grade:  0-1  0-2  0-3  0-4  0-5  0-6 


Aeronautical  Rating: 

Transports: 

First  Pilot 

Aircraft  Commander 
Instructor  Pilot 
Evaluator  Pilot 
Other  (Please  specify) 


Fighters: 

Pilot 

Element  Lead 
Flight  Lead 
Instructor  Pilot 
Evaluator  Pilot 


Commercial: 

First  Officer 
Captain 
Check  Airman 


Please  list  the  aircraft  flown  and  the  approximate  number  of  hours  in  each,  begiiming  with  the 
most  recent. 

Aircraft:  Hours: 


Grand  Total:  _ 

Indicate  your  experience  level  with  the  following: 

a)  Precision  approaches  in  low  visibility  conditions 

Transport  pilots:  Weather  certification  (CAT  II,  CAT  III) 
Fighter  pilots:  Pilot  weather  category  (ceiling/visibility) 

b)  HUD  instrument  approaches  (none,  a  few,  several,  etc.)  _ 

c)  Approximate  number  of  hours  flown  with  electro-optical  sensors 
such  as  IR,  radar,  low  light  TV,  etc. 


Organization: _ 

Duty  Station: _ ^Duty  Phone: 

May  we  contact  you  if  additional  information  is  needed?  _ 

Time  Since  Last  Flight:  Months/Days  _ 
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The  Full  HUD  Design 


Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the  tables 
below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the  Full  HUD  in  the 
performance  of  each  of  these  tasks  for  each  of  the  out-the-window  (OTW)  conditions 
identified.  Please  provide  comments  and  suggested  design  alternatives  for  any  item  rated 
“borderline  or  worse”  (i.e.,  C,  D,  or  E).  The  first  table  applies  to  the  approach  segment  before 
the  sensor  image  was  available.  The  second  table  applies  to  the  approach  segment  after  the 
sensor  image  was  available. 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor  issues 
having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation;  changes  would 
be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations  impair 
pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to  mission 
failure;  redesign  required. 


1.  3efore  the  sensor  image/visual  scene  was  available: 


_ — - - - - - 

a)  Safely  fly  the  approach 

/ 

b)  Monitor  an^;!  control  lateral  i^eviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

Comments; 
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a)  3afely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

0)  Determine  desired  touchdoivn  point 

Comments: 


3.  Use  the  above  Acceptability  Scale  to  rate  the  overall  acceptability  of  the  Full  HUD. _ 

4.  Use  the  Modified  Cooper  Harper  Scale  (last  page  of  this  package)  to  rate  the  workload 
associated  with  the  Full  HUD  in  each  of  the  following  conditions: 

a.  VMC _ 

b.  With  a  sensor  image _ 

c.  With  the  comer  reflectors _ 

d.  Without  a  sensor  image _ 
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5.  Did  you  experience  any  clutter  problems  with  the  Full  HUD: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


6.  Was  there  any  necessary  information  missing  from  the  Full  HUD: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


7.  Was  there  any  unnecessary  information  provided  by  the  full  HUD: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


8.  Were  there  any  safety  issues  associated  with  the  Full  HUD: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 
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9.  What  strategy  did  you  incorporate  for  using  the  Full  HUD  in  the  Precision  Approach  Task? 


10.  Please  provide  any  additional  comments  or  design  alternatives  that  you  may  have  regarding 
the  Full  HUD. 
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The  Basic  HUD  with  the  Runway  Symbol  Design 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the  tables 
below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the  Basic  HUD  with 
the  Runway  Symbol  design  in  the  performance  of  each  of  these  tasks  for  each  of  the  out-the- 
window  (OTW)  conditions  identified.  Please  provide  comments  and  suggested  design 
alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E).  The  first  table  applies 
to  the  approach  segment  before  the  sensor  image  was  available.  The  second  table  applies  to 
the  approach  segment  after  the  sensor  image  was  available. 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor  issues 
having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation;  changes  would 
be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations  impair 
pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to  mission 
failure;  redesign  required. 


1.  before  the  eeneor  ima^e/visual  ecene  wae  available: 


a)  5afely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alont^  the  approach  path 

Comments: 
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2.  After  the  sensor  image/visual  scene  was  available; 


a)  Safely  fly  the  approach 

b)  Monitorand  control  lateral  deviation 

c)  Monitorand  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

0)  Determine  desired  touchdown  point 

Comments: 


3.  Use  the  above  Acceptability  Scale  to  rate  the  overall  acceptability  of  the  Basic  HUD  with  the 

Runway  Symbol  design. _ 

4.  Use  the  Modified  Cooper  Harper  Scale  (last  page  of  this  package)  to  rate  the  workload 
associated  with  the  Basic  HUD  with  the  Runway  Symbol  design  in  each  of  the  following 
conditions: 

a.  VMC _ 

b.  With  a  sensor  image _ 

c.  With  the  comer  reflectors _ 

d.  Without  a  sensor  image _ 
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5.  Did  you  experience  any  clutter  problems  with  the  Basic  HUD  with  the  Runway  Symbol 
design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


6.  Was  there  any  necessary  information  missing  from  the  Basic  HUD  with  the  Runway  Symbol 
design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


7.  Was  there  any  unnecessary  information  provided  by  the  Basic  HUD  with  the  Runway  Symbol 
design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


8.  Were  there  any  safety  issues  associated  with  the  Basic  HUD  with  the  Runway  Symbol  design: 
a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 
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9.  What  strategy  did  you  incorporate  for  using  the  Basic  HUD  with  the  Runway  Symbol  design 
in  the  Precision  Approach  Task? 


10.  Please  provide  any  additional  comments  or  design  alternatives  that  you  may  have  regarding 
the  Basic  HUD  with  the  Rimway  Symbol  design. 
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The  Basic  HUD  with  the  Raw  Deviation  Data  Design 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the  tables 
below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the  Basic  HUD  with 
the  Raw  Deviation  Data  design  in  the  performance  of  each  of  these  tasks  for  each  of  the  out- 
the-window  (OTW)  conditions  identified.  Please  provide  comments  and  suggested  design 
alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E).  The  first  table  applies 
to  the  approach  segment  before  the  sensor  image  was  available.  The  second  table  applies  to 
the  approach  segment  after  the  sensor  image  was  available. 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor  issues 
having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation;  changes  would 
be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations  impair 
pilot  performeince;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to  mission 
failure;  redesign  required. 


1.  &eforet)ne  sensor  image/visual  scene  was  available: 


a)  5afaly  fly  tha  approach 

b)  Monitorand  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

Comments; 
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a)  Safely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

q)  Determine  desired  touchdown  point 

Comments: 


3.  Use  the  above  Acceptability  Scale  to  rate  the  overall  acceptability  of  the  Basic  HUD  with  the 

Raw  Deviation  Data  design. _ 

4.  Use  the  Modified  Cooper  Harper  Scale  (last  page  of  this  package)  to  rate  the  workload 
associated  with  the  Basic  HUD  with  the  Raw  Deviation  Data  design  in  each  of  the  following 
conditions: 

a.  VMC _ 

b.  With  a  sensor  image _ 

c.  With  the  comer  reflectors _ 

d.  Without  a  sensor  image _ 
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5.  Did  you  experience  any  clutter  problems  with  the  Basic  HUD  with  the  Raw  Deviation  Data 
design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


6.  Was  there  any  necessary  information  missing  from  the  Basic  HUD  with  the  Raw  Deviation 
Data  design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


7.  Was  there  any  unnecessary  information  provided  by  the  Basic  HUD  with  the  Raw  Deviation 
Data  design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


8.  Were  there  any  safety  issues  associated  with  the  Basic  HUD  with  the  Raw  Deviation  Data 
design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 
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9.  What  strategy  did  you  incorporate  for  using  the  Basic  HUD  with  the  Raw  Deviation  Data 
design  in  the  Precision  Approach  Task? 


10.  Please  provide  any  additional  comments  or  design  alternatives  that  you  may  have  regarding 
the  Basic  HUD  with  the  Raw  Deviation  Data  design. 
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The  Basic  HUD  with  Both  the  Runway  Symbol  and  Raw  Deviation  Design 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  imique  tasks  in  the  tables 
below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the  Basic  HUD  with 
both  the  Runway  Symbol  and  Raw  Deviation  Data  design  in  the  performance  of  each  of 
these  tasks  for  each  of  the  out-the-window  (OTW)  conditions  identified.  Please  provide 
comments  and  suggested  design  alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C, 
D,  or  E).  The  first  table  applies  to  the  approach  segment  before  the  sensor  image  was 
available.  The  second  table  applies  to  the  approach  segment  after  the  sensor  image  was 
available. 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor  issues 
having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation;  changes  would 
be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations  impair 
pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to  mission 
failure;  redesign  required. 


1.  3eforet\^e  sensor  image/visual  scene  was  available: 


a)  Safely  fly  the  approach 

b)  Monitoran^  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

Comments: 
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2.  After  the  sensor  image/visual  scene  was  available; 


a)  6af(5iy  fiy  the  approach 

b)  Monitorand  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

Determine  desired  touchdown  point 

Comments; 


3.  Use  the  above  Acceptability  Scale  to  rate  the  overall  acceptability  of  the  Basic  HUD  with 

both  the  Rimway  Symbol  and  Raw  Deviation  Data  design. _ 

4.  Use  the  Modified  Cooper  Harper  Scale  (last  page  of  this  package)  to  rate  the  workload 
associated  with  the  Basic  HUD  with  both  the  Runway  Symbol  and  Raw  Deviation  Data 
design  in  each  of  the  following  conditions: 

a.  VMC _ 

b.  With  a  sensor  image _ 

c.  With  the  comer  reflectors _ 

d.  Without  a  sensor  image _ 
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5.  Did  you  experience  any  clutter  problems  with  the  Basic  HUD  with  both  the  Rimway  Symbol 
and  Raw  Deviation  Data  design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


6.  Was  there  any  necessary  information  missing  from  the  Basic  HUD  with  both  the  Runway 
Symbol  and  Raw  Deviation  Data  design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


7.  Was  there  any  unnecessary  information  provided  by  the  Basic  HUD  with  both  the  Runway 
Symbol  and  Raw  Deviation  Data  design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 


8.  Were  there  any  safety  issues  associated  with  the  Basic  HUD  with  both  the  Runway  Symbol 
and  Raw  Deviation  Data  design: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 


b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 
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9.  What  strategy  did  you  incorporate  for  using  the  Basic  HUD  with  both  the  Runway  Symbol 
and  Raw  Deviation  Data  design  in  the  Precision  Approach  Task? 


10.  Please  provide  any  additional  comments  or  design  alternatives  that  you  may  have  regarding 
the  Basic  HUD  with  both  the  Runway  Symbol  and  Raw  Deviation  Data  design. 
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Some  General  Questions 


The  Experimental  HUD  Configurations 

1 .  Overall,  which  HUD  configuration  did  you  most  prefer?  Why? 

2.  Overall,  which  HUD  configuration  did  you  least  prefer?  Why? 


3.  Did  you  experience  any  problems  associated  with  the  design  and  mechanization  of  any  of  the 
individual  HUD  symbols?  If  so,  please  explain. 
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The  TRAC  Simulator 


Instructions:  Use  the  scale  below  to  rate  the  acceptability  of  the  following  simulator  features. 

A.  Very  acceptable:  A  good  simulation  as  is. 

B.  Moderately  acceptable:  Minor  simulation  deficiencies  exist  that  do  not  impact  pilot 
performance. 

C.  Borderline:  Simulation  deficiencies  exist  that  could  impact  pilot  performance; 
improvements  would  be  desirable. 

D.  Moderately  unacceptable:  Simulation  deficiencies  degrade  pilot  performance; 
improvements  required. 

E.  Very  unacceptable:  Simulation  is  too  deficient  to  provide  the  intended  fimction. 

1 .  Flight  characteristics _ 

2.  Throttle  characteristics _ 

3.  Out-the-Window  display _ 

4.  Sensor  image  representation _ 

5.  HUD  representation _ 

6.  Cockpit  geometry _ 


7.  Were  you  able  to  adapt  to  the  simulator  aeromodel  in  a  reasonable  period? 

Yes _ No _ .  If  No,  please  comment. 

8.  Were  the  random  winds  adequate  at  inducing  a  nominal  amount  of  pilot  control  in  order  to 

remain  on  course  and  glideslope?  Yes _ No _ . 

9.  Was  the  Out-the-Window  scene  commensurate  with  your  experience  with  approaches  in  low 

visibility  conditions?  Yes _ No _ .  If  No,  please  comment. 


The  ALG  Concept 

1 .  Do  you  think  there  is  a  future  for  ALG  in  the  major  commercial  passenger  and  freight 

industry?  Yes _ No _ .  Comments: 

2.  Using  an  ALG  display  similar  to  the  one  in  this  evaluation,  would  you  be  comfortable  flying 

low  visibility  approaches?  Yes _ No _ .  Comments; 
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ALG  PROGRAM  CRITIQUE 


1.  The  billeting  and  other  accommodations  during  your  stay  were:  poor  fair  good 

Comments: _ 


2.  Generally  speaking,  the  TRAC  test  facility  (simulation  bay,  pilots’  briefing  room,  etc.)  was: 

poor  fair  good 


Comments: 


3.  Please  rate  the  experimenters’:  knowledge 

poor 

fair 

good 

professionalism 

poor 

fair 

good 

preparedness 

poor 

fair 

good 

Comments: 


4.  Please  evaluate  the  following  components  of  the  ALG  program: 


a)  ground  training  as  a  whole 

introduction  and  administration 

poor 

fair 

good 

(groimd  training  session  one) 
cockpit  layout 

poor 

fair 

good 

(groimd  training  session  two) 

HUD  format  and  symbology 

poor 

fair 

good 

(ground  training  session  three) 

poor 

fair 

good 

Comments: 
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b)  flight  training  as  a  whole 

poor 

fair 

good 

simulator/display  familiarization 

good 

(flight  training  session  one) 

poor 

fair 

proficiency  verification 

fair 

good 

(flight  training  session  two) 

poor 

Comments 


c. 

data  collection  process  as  a  whole 

poor 

fair 

good 

data  collection  sessions 

poor 

fair 

good 

post-data  collection  surveys  and  forms 

poor 

fair 

good 

Comments: 


5.  What  did  you  like  the  most  about  participating  in  the  ALG  study? 


6.  What  did  you  like  the  least  about  participating  in  the  ALG  study? 
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Modified  Cooper-Harper  Scale 
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APPENDIX  B  -  ANALYSIS  OF  VARIANCE  RESULTS 


74 


STATISTICS  SUMMARY 


1.  Introduction 

The  results  of  the  analysis  procedures  performed  on  the  Precision  Approach  Task  and 
workload  data  are  provided  in  this  Appendix.  For  each  variable,  the  first  table  provides  the 
results  of  ad-hoc  Analysis  of  Variance  (ANOVA)  procedures  and  includes  F  statistics, 
degrees  of  freedom,  and  significance  levels  for  each  main  and  interaction  effect.  Results  of 
post-hoc  ANOVAs  and  Duncan  tests  are  also  provided.  Finally,  the  means  for  all  test 
conditions  are  provided. 

2.  Precision  Approach  Task  Performance  Data 

The  analyses  treated  in  this  section  pertain  to  the  data  collected  in  the  Precision 
Approach  Task.  The  analyses  will  be  presented  separately  for  each  dependent  variable. 

2.1  Root  Mean  Square  Glideslope  Deviation 

A  4x4x2  ANOVA  (4  head-up  displays  (HUDs),  4  SensorA^isibility  Conditions,  and  2 
Approach  Segments)  was  performed  on  the  Root  Mean  Square  (RMS)  glideslope  deviation 
data.  The  results  are  provided  in  Table  B-1 . 


Table  B-1.  ANOVA  Results  for  RMS  Glideslope  Deviation 


Tested  Effect 

ANOVA  Results 

HUD 

F(3,33)  =  52.87;  p  =  .000 

Approach  Segment 

F(l,ll)=  110.41;  p  =  .000 

SensorA^isibility 

F(3,33)  =  0.22;  p  =  .347 

HUD  X  Approach  Segment 

F(3,33)  =  19.99;  p  =  .000 

HUD  X  Sensor A^isibility 

F(9,99)  =  2.99;  p  =  .003 

Approach  Segment  x  SensorA^isibility 

F(3,33)  =  0.23;  p=.877 

HUD  x  Approach  Segment  x  Sensor A^isibility 

F(9,99)  =  2.51;  p  =  .012 

The  significant  three-way  interaction  between  HUD  Symbology  Set,  Approach  Segment, 
and  SensorA^isibility  Condition  was  treated  by  testing  the  HUD  Symbology  Set  by 
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SensorA^isibility  Condition  interaction  separately  for  each  Approach  Segment  (Long  and 
Short  Final).  This  interaction  was  significant  for  the  Short  Final  Approach  Segment,  F(9,99) 
=  3.10;  p  =  .003,  but  not  for  the  Long  Final  Approach  Segment,  F(9,99)  =  0.95;  p  =  .484. 
Post-hoc  one-way  ANOVAs  were  applied  to  the  Short  Final  Approach  Segment  data  to  test 
for  performance  differences  across  HUD  Symbology  Sets  for  each  SensorA^isibility 
Condition.  The  results  of  the  ANOVAs  are  shown  in  Table  B-2.  Additional  post-hoc  one¬ 
way  ANOVAs  were  applied  to  the  Short  Final  Approach  Segment  data  to  test  for 
performance  differences  across  the  SensorA^isibility  Conditions  for  each  HUD  Symbology 
Set.  The  results  of  these  tests  are  shown  in  Table  B-3. 


Table  B-2.  Post-Hoc  RMS  Glideslope  Analyses 


Tested  Effect 

ANOVA  Results 

HUD  X  (VMC;  Short  Final) 

F(3,33)  =  15.64;  p  =  .000 

HUD  X  (FLIR;  Short  Final) 

F(3,33)  =  27.15;  p  =  .000 

HUD  X  (Comer  Reflectors;  Short  Final) 

F(3,33)=  14.03;  p  =  .000 

HUD  X  (No  Sensor;  Short  Final) 

F(3,33)  =  41.50;  p  =  .000 

Table  B-3.  Post-Hoc  RMS  Glideslope  Analyses 


Tested  Effect 

ANOVA  Results 

Sensor/Visibility  x  (Basic/RWY,  Short  Final) 

F(3,33)  =  3.78;  p  =  .020 

Sensor/Visibility  x  (Basic/RAW,  Short  Final) 

F(3,33)  =  0.80;  p  =  .505 

Sensor/Visibility  x  (Basic/BOTH,  Short  Final) 

F(3,33)  =  0.15;  p  =  .930 

Sensor/Visibility  x  (Full  HUD,  Short  Final) 

F(3,33)  =  0.67;  p=-.578 

The  mean  glideslope  deviations  are  shown  for  each  Approach  Segment  in  Tables  B-4 
and  B-5.  Duncan  Multiple  Range  tests  were  conducted  on  the  means  associated  with  the 
significant  one-way  tests.  The  results  of  the  Duncan  tests  indicated  that  during  the  Short 
Final  Approach  Segment  with  the  Basic/RWY  HUD  Symbology  Set,  RMS  glideslope 
performance  with  the  FLIR  and  No  Sensor  SensorA/^isibility  Conditions  differed  significantly 
from  the  VMC  Condition.  In  addition,  the  values  of  RMS  glideslope  during  the  Short  Final 
Approach  Segment  with  the  Basic/RWY  HUD  and  the  No  Sensor  Sensor/Visibility 
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Condition  were  significantly  different  from  the  Comer  Reflector  Condition.  The  Duncan 
tests  also  indicated  that  during  the  Short  Final  Approach  Segment  in  all  of  the 
SensorA^isibility  Conditions,  the  RMS  glideslope  deviations  with  the  Basic/RWY  HUD  were 
significantly  larger  than  those  for  the  other  three  HUD  Symbology  Sets. 


Table  B-4.  Means  Used  in  Post-Hoc  RMS  Glideslope  Analyses 


1  Long  Final  ^ 

VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

.560 

.617 

.608 

.702 

.622 

Basic/RAW 

.094 

,124 

.075 

.103 

.099 

Basic/BOTH 

.101 

.098 

.104 

.108 

.102 

Full 

.137 

.117 

.152 

.137 

.136 

Overall 

.224 

.239 

.235 

.264 

.241 

Table  B-5.  Means  Used  in  Post-Hoc  RMS  Glideslope  Analyses 


1  Short  Final  | 

VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

1.423 

1.821 

1.561 

1.956 

1.69 

Basic/RAW 

0.620 

0.601 

0.678 

0.478 

.594 

Basic/BOTH 

0.654 

0.636 

0.635 

0.573 

.624 

Full 

0.853 

0.780 

0.875 

0.724 

.808 

Overall 

.890 

.960 

.938 

.935 

.931 

2.2  RMS  Localizer  Deviation 

A  4x4x2  ANOVA  (4  HUDs,  4  SensorA^isibility  Conditions,  and  2  Approach  Segments) 
was  performed  on  the  RMS  localizer  deviation  data.  The  results  are  provided  in  Table  B-6. 
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Table  B-6.  AN OVA  Results  for  RMS  Localizer  Deviation 


Tested  Effect 

ANOVA  Result 

HUD 

F(3,33)  =  5.84;  p  =  .003 

Approach  Segment 

F(l,ll)  =  8.21;  p  =  .615 

Sensor/Visibility 

F(3,33)=  1.74;  p=.177 

HUD  X  Approach  Segment 

F(3,33)  =  2.44;  p  =  .081 

HUD  X  SensorA^isibility 

F(9,99)  =  0.92;  p  =  .513 

Approach  Segment  x  Sensor A^isibility 

F(3,33)  =  6.86;  p  =  .001 

HUD  X  Approach  Segment  x  Sensor/Visibility 

F(9,99)  =  1.53;  p=.146 

A  post-hoc  ANOVA  was  performed  on  the  RMS  localizer  deviation  data  for  the 
significant  interaction  between  the  Approach  Segment  and  SensorA^isibility  conditions.  The 
results  showed  a  significant  main  effect  of  the  SensorA^isibility  Condition  for  the  Short  Final 
Approach  Segment,  F(3,33)  =  12.08;  p  =  .000,  but  not  for  the  Long  Final  Approach 
Segment,  F(3,33)  =  2.25;  p  =  .101.  A  Duncan  test  indicated  that  RMS  localizer  performance 
in  the  VMC  SensorA/^isibility  Condition  differed  significantly  from  that  in  the  No  Sensor 
Condition  during  the  Short  Final  Approach  Segment.  The  means  for  this  analysis  are  shown 
in  Table  B-7. 


Table  B-7.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Localizer  Deviation 


Long  Final 

Short  Final 

VMC 

0.122 

0.077 

FLIR 

0.104 

0.100 

Comer  Reflectors 

0.108 

0.099 

No  Sensor 

0.120 

0.128 

A  Duncan  test  was  also  applied  to  the  main  effect  of  the  HUD  Symbology  Set.  The 
results  of  the  Duncan  test  showed  that  RMS  localizer  deviations  with  the  Full  HUD  and 
Basic/RWY  HUD  Symbology  Sets  were  significantly  larger  than  those  for  the  Basic/RAW 
and  the  Basic/BOTH  HUD  Symbology  Sets.  Means  for  the  different  HUD  Symbology  Sets 
as  a  function  of  Sensor A^isibility  Condition  are  shown  in  Tables  B-8  and  B-9  for  the  Long 
and  Short  Final  Approach  Segments  respectively. 
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Table  B-8.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Localizer  Deviation 


1  Long  Final  | 

VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

.117 

.146 

.145 

.153 

.140 

Basic/RAW 

.087 

.082 

.092 

.105 

.092 

Basic/BOTH 

.124 

.089 

.096 

.106 

.103 

Full 

.161 

.097 

.099 

.116 

.118 

Overall 

.122 

.104 

.108 

.120 

.113 

Table  B-9.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Localizer  Deviation 


1  Short  Final 

VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

.073 

.111 

.106 

.121 

.103 

Basic/RAW 

.071 

.091 

.080 

.116 

.089 

Basic/BOTH 

.076 

.085 

.078 

.119 

.090 

Full 

.089 

.115 

.132 

.150 

.121 

Overall 

.077 

.100 

.099 

.126 

.101 

2.3  Vertical  Velocity 

A  4x4x2  ANOVA  (4  HUDs,  4  SensorA^isibility  Conditions,  and  2  Approach  Segments) 
was  performed  on  the  RMS  vertical  velocity  data.  The  results  are  provided  in  Table  B-10. 


Table  B-10.  ANOVA  Results  for  RMS  Vertical  Velocity 


Tested  Effect 

ANOVA  Result 

HUD 

F(3,33)=  16.60;  p  =  .000 

Approach  Segment 

F(l,ll)  =  69.49;  p  =  .000 

Sensor/Visibility 

F(3,33)  =  0.85;  p  =  .478 

HUD  X  Approach  Segment 

F(3,33)  =  23.31;  p  =  .000 

HUD  X  Sensor/Visibility 

F(9,99)  =  3.13;  p  =  .002 

Approach  Segment  x  Sensor/Visibility 

F(3,33)  =  3.01;  p  =  .044 

HUD  X  Approach  Segment  x  Sensor/Visibility 

F(9,99)  =  0.51;  p  =  .863 

The  significant  interaction  between  the  HUD  Symbology  Set  and  the  Approach  Segment 
for  RMS  vertical  velocity  was  investigated  with  a  post-hoc  one-way  ANOVA  testing  for 
differences  across  the  HUD  Symbology  Set  for  the  Long  and  Short  Final  Approach 
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Segments.  The  results  showed  a  significant  main  effect  for  the  HUD  Symbology  Set  during 
the  Short  Final  Approach  Segment,  F(3,33)  =  23.68;  p  =  .000,  but  not  during  the  Long  Final 
Approach  Segment,  F(3,33)  =  1.00;  p  =  .406.  A  Duncan  test  showed  that  the  values  of 
vertical  velocity  for  the  Basic/RWY  HUD  Symbology  Set  were  significantly  different  from 
each  of  the  other  HUD  Symbology  Sets.  In  addition,  this  Duncan  test  revealed  a  significant 
difference  in  the  values  of  vertical  velocity  between  the  Basic/RAW  HUD  Symbology  Set 
and  the  Full  HUD. 

The  significant  interaction  of  the  HUD  Symbology  Set  and  the  SensorA^isibility 
Condition  on  RMS  vertical  velocity  was  investigated  with  a  series  of  post-hoc  one-way 
ANOVAs.  The  results  are  provided  in  Table  B-1 1.  The  effect  of  the  HUD  Symbology  Set  as 
a  function  of  the  SensorA^isibility  Condition  was  significant  for  the  FLIR,  Comer  Reflectors, 
and  No  Sensor  conditions.  The  differences  among  HUD  Symbology  Sets  were  investigated 
with  a  post-hoc  Duncan  test,  which  indicated  that  vertical  velocity  deviations  were  smallest 
with  the  Basic/RWY  Symbology  Set  for  all  Sensor/Visibility  Conditions.  The  means  used  in 
these  evaluation  are  provided  in  Tables  B-1 2,  B-1 3,  and  B-1 4. 

Table  B-11.  Post-Hoc  RMS  Vertical  Velocity  Analysis  -  HUD  Symbology  Set  by 
Sensor/Visibility  Condition  Interaction 


Tested  Effect 

ANOVA  Result 

HUD  (VMC) 

F(3,33)=  1.94;  p  =  .143 

HUD  (FLIR) 

F(3,33)  =  7.60;  p  =  .001 

HUD  (Comer  Reflectors) 

F{3,33)  =  7.63;  p  =  .001 

HUD  (No  Sensor) 

F(3,33)=  18.41;  p  =  .000 

Table  B-1 2.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Vertical  Velocity 


1  Long  Final  1 

VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

1150.72 

1133.67 

1137.76 

1116.31 

1134.62 

Basic/RAW 

1117.65 

1125.66 

1131.41 

1129.22 

1125.98 

Basic/BOTH 

1 125.06 

1124.32 

1139.12 

1115.51 

1126.01 

Full 

1141.03 

1120.40 

1131.07 

1132.04 

1131.13 

Overall 

1133.71 

1126.01 

1134.88 

1123.18 

1129.44 
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Table  B-13.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Vertical  Velocity 


1  Short  Final  | 

VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

1033.89 

993.88 

1004.63 

1005.21 

1009.40 

Basic/RAW 

1089.15 

1105.22 

1100.74 

1140.06 

1108.79 

Basic/BOTH 

1097.04 

1092.54 

1096.31 

1094.66 

1095.12 

Full 

1078.43 

1077.88 

1075.56 

1092.24 

1080.96 

Overall 

1074.39 

1067.37 

1069.24 

1082.95 

1073.47 

Table  B-14.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Vertical  Velocity 


VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Basic/RWY 

1092.30 

1063.78 

1071.19 

1060.76 

Basic/RAW 

1103.40 

1115.44 

1116.07 

1134.64 

Basic/BOTH 

1111.05 

1108.43 

1117.71 

1105.09 

Full 

1109.73 

1099.14 

1103.32 

1112.06 

The  significant  interaction  between  the  Approach  Segment  and  the  SensorA^isibility 
Condition  for  RMS  vertical  velocity  was  investigated  with  a  post-hoc  ANOVA.  No 
significant  effects  were  found  for  the  Long  Final,  F(3,33)  =  1.64;  p  =  .199,  or  Short  Final 
Approach  Segments,  F(3,33)  =  2.47;  p  =  .079. 

2.4  Indicated  Airspeed 

A  4x4x2  ANOVA  (4  HUDs,  4  SensorA^isibility  Conditions,  and  2  Approach  Segments) 
was  performed  on  the  RMS  indicated  airspeed  (IAS)  data.  The  results  are  provided  in  Table 
B-15. 


Table  B-15.  ANOVA  Results  for  RMS  Indicated  Airspeed 


Tested  Effect 

ANOVA  Result 

HUD 

F(3,33)  =  2.45;  p  =  .081 

Approach  Segment 

F(l,ll)  =  0.10;  p  =  .753 

Sensor/Visibility 

F(3,33)  =  3.58;  p  =  .024 

HUD  X  Approach  Segment 

F(3,33)  =  4.18;  p=.013 

HUD  X  Sensor/Visibility 

F(9,99)  =  2.46;  p=.014 

Approach  Segment  x  Sensor/Visibility 

F(3,33)  =  0.34;  p  =  .796 

HUD  X  Approach  Segment  x  Sensor/Visibility 

F(9,99)  =  0.50;  p  =  .872 
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The  significant  interactions  of  the  HUD  Symbology  Set  and  the  Approach  Segment  on 
RMS  IAS,  as  well  as  HUD  Symbology  Set  and  SensorA^isibility  Condition  on  RMS  IAS, 
were  investigated  with  post-hoc  ANOVAs.  The  results  of  the  analysis  involving  the 
interaction  of  the  HUD  Symbology  Set  and  the  Approach  Segment  revealed  a  significant 
main  effect  of  HUD  Symbology  Set  for  both  the  Long  Final,  F(3,33)  =  3.14;  p  =  .038,  and 
Short  Final,  F(3,33)  =  2.98;  p  =  .045  Approach  Segments.  For  the  Long  Final  Approach 
Segment,  Duncan  tests  showed  that  RMS  IAS  deviations  were  the  smallest  for  the 
Basic/RWY  HUD  Symbology  Set.  For  the  Short  Final  Approach  Segment,  RMS  IAS 
deviations  were  smaller  for  the  Basic/RWY  and  Basic/RAW  Symbology  Sets  than  for  the 
Basic/BOTH  and  Full  HUD  Symbology  Sets. 

The  post-hoc  analysis  of  the  HUD  Symbology  Set  by  the  Sensor/Visibility  Condition 
interaction  are  shown  in  Table  B-16.  As  the  table  shows,  a  significant  main  effect  for  HUD 
Symbology  Set  was  found  for  both  the  VMC  and  No  Sensor  Conditions.  For  the  VMC 
Condition,  a  Duncan  test  showed  that  RMS  IAS  deviations  were  larger  for  the  Full  HUD 
Symbology  Set  than  for  the  Basic/RAW  and  Basic/BOTH  Symbology  Sets.  For  the  No 
Sensor  Condition,  a  Duncan  test  showed  that  RMS  IAS  deviations  for  both  the  Basic/RAW 
and  Full  HUD  Symbology  Sets  were  significantly  larger  than  for  the  Basic/Both  Symbology 
Set. 


Table  B-16.  Post-Hoc  RMS  Indicated  Airspeed  Analysis  -  HUD  Symbology  Set  by 
Sensor/Visibility  Condition  Interaction 


Tested  Effect 

ANOVA  Result 

HUD  (VMC) 

F(3,33)  =  3.16;  p  =  .038 

HUD  (FUR) 

F(3,33)  =  0.75;  p  =  .532 

HUD  (Comer  Reflectors) 

F(3,33)  =  2.26;  p  =  .100 

HUD  (No  Sensor) 

F(3,33)  =  3.64;  p  =  .023 

The  means  used  in  the  above  analyses  are  provided  in  Table  B-17,  B-18,  and  B-19. 
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Table  B-1 7.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Indicated  Airspeed 


1  Long  Final  ] 

VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

3.68 

3.01 

3.25 

3.32 

3.315 

Basic/RAW 

3.37 

3.80 

4.23 

4,79 

4.049 

Basic/BOTH 

3.93 

4.13 

4.42 

3.75 

4.057 

Full 

4.13 

3.48 

3.42 

4.75 

3.941 

Overall 

3.775 

3.603 

3.832 

4.149 

3.839 

Table  B-1 8.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Indicated  Airspeed 


1  Short  Final  | 

VMC 

FUR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

3.87 

3.62 

3.77 

4.55 

3.951 

Basic/RAW 

3.34 

3.77 

4.77 

4.80 

4.173 

Basic/BOTH 

3.19 

3.59 

3.51 

3.37 

3.418 

Full 

4.39 

4.06 

4.01 

4.76 

4.302 

Overall 

3.702 

3.761 

4.016 

4.367 

3.961 

Table  B-1 9.  Means  Used  in  the  Post-Hoc  Analysis  of  RMS  Indicated  Airspeed 


VMC 

FLIR 

Comer 

Reflectors 

No  Sensor 

Basic/RWY 

3.78 

3.31 

3.51 

3.93 

Basic/RAW 

3.35 

3.79 

4.50 

4.80 

Basic/BOTH 

3.56 

3.86 

3.96 

3.56 

Full 

4.26 

3.77 

3.71 

4.61 

2.5  Touchdown  Deviation 

A  4x4  ANOVA  (4  HUDs  by  4  SensorA^isibility  Conditions)  was  done  on  the  touchdown 
deviation  from  the  runway  point  of  intercept  (RPI)  data.  See  Table  B-20  for  the  results. 


Table  B-20.  ANOVA  Results  for  Touchdown  Deviation  from  RPI  Data 


Tested  Effect 

ANOVA  Result 

HUD 

F(3,33)  =  3.48;  p  =  .027 

Sensor/Visibility 

F(3,30)  =  0.13;  p  =  .941 

HUD  X  Sensor/Visibility 

F(9,89)  =  2.01;  p  =  .046 
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The  significant  two-way  interaction  between  the  HUD  Symbology  Set  and  the 
SensorA^isibility  Condition  was  treated  by  using  post-hoc  one-way  ANOVAs  to  test  for  RPI 
touchdown  deviation  differences  across  HUD  Symbology  Sets  for  each  SensorA^isibility 
Condition.  The  results  of  these  post-hoc  ANOVAs  are  shown  in  Table  B-21.  Post-hoc  one¬ 
way  ANOVAs  were  also  used  to  test  for  RPI  touchdown  deviation  differences  across 
SensorA^isibility  Conditions  for  each  HUD  Symbology  Set.  The  results  of  these  post-hoc 
ANOVAs  are  shown  in  Table  B-22.  Significant  differences  across  HUD  Symbology  Sets 
were  only  detected  for  the  VMC  SensorA^isibility  Condition.  A  Duncan  test  indicated  that 
RPI  touchdown  deviations  were  larger  with  the  Full  HUD  Symbology  Set  than  for  any  of  the 
other  HUD  Symbology  Sets.  Since  no  significant  differences  were  foimd  across  the 
SensorA/'isibility  Conditions  for  any  of  the  HUD  Symbology  Sets,  no  further  analysis  was 
completed. 


Table  B-21.  Post-Hoc  RPI  Touchdown  Deviation  Analyses 


Tested  Effect 

ANOVA  Results 

HUD  X  (VMC) 

F(3,33)  =  3.81;  p  =  .019 

HUD  X  (FLIR) 

F(3,33)  =  0.93;  p  =  .439 

HUD  X  (Comer  Reflectors) 

F(3,33)  =  2.23;  p  =  .103 

HUD  X  (No  Sensor) 

F(3,33)  =  0.18;  p  =  .910 

Table  B-22.  Post-Hoc  RPI  Touchdown  Deviation  Analyses 


Tested  Effect 

ANOVA  Results 

SensorA^isibility  x  (Basic/RWY) 

F(3,33)=  1.27;  p  =  .301 

SensorA^isibility  x  (Basic/RAW) 

F(3,33)  =  0.69;  p  =  .566 

SensorA^isibility  x  (Basic/BOTH) 

F(3,33)  =  1.24;  p  =  .311 

Sensor/Visibility  x  (Full  HUD) 

F(3,33)  =  2.74;  p  =  .060 

The  means  used  in  this  analysis  are  provided  in  Table  B-23. 
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Table  B-23.  Means  Used  in  the  Analysis  of  RPI  Touchdown  Deviation 


VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

278.29 

382.60 

350.23 

407.39 

349.73 

Basic/RAW 

290.04 

253.30 

230.52 

286.94 

264.88 

Basic/BOTH 

296.57 

251.26 

300.77 

254.61 

276.07 

Full 

409.62 

383.43 

389.22 

295.27 

371.44 

Overall 

316.91 

316.89 

314.87 

305.45 

2.6  Centerline  Deviation 

A  4x4  ANOVA  (4  HUDs  and  4  SensorA^isibility  Conditions)  was  performed  on  the 
centerline  deviation  data.  The  results  are  provided  in  Table  B-24.  A  significant  main  effect 
was  fmmd  for  SensorA/’isibility  Conditions.  A  post-hoc  Duncan  test  showed  that  centerline 
deviations  for  the  Comer  Reflector  and  No  Sensor  Conditions  were  significantly  larger  than 
for  those  in  the  VMC  and  FLIR  Conditions.  The  means  used  in  this  evaluation  are  provided 
in  Table  B-25. 


Table  B-24.  ANOVA  Results  for  Centerline  Deviation  Data 


Tested  Effect 

ANOVA  Result 

HUD 

F(3,33)  =  0.75;  p  =  .532 

SensorA^isibility 

F(3,33)  =  7.05;  p  =  .001 

HUD  X  Sensor A^isibility 

F(9,98)  =  0.68;  p  =  .723 

Table  B-25.  Means  Used  in  the  Analysis  of  Centerline  Deviation 


VMC 

FLIR 

Comer  Reflectors 

No  Sensor 

Overall 

Basic/RWY 

12.37 

13.07 

26.63 

18.62 

17.49 

Basic/RAW 

12.31 

13.14 

15.33 

17.51 

14.46 

Basic/BOTH 

11.38 

13.45 

21.46 

17.17 

15.90 

Full 

17.32 

12.24 

20.60 

18.14 

17.01 

Overall 

13.29 

12.98 

20.96 

17.80 
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3.  Workload  Data 


A  4x2  ANOVA  (4  HUDs  and  2  Approach  Segments)  was  performed  on  the  workload 
ratings  gathered  with  the  Subjective  WORkload  Dominance  (SWORD)  technique.  The 
results  are  provided  in  Table  B-26. 


Table  B-26.  ANOVA  Results  for  Workload  Ratings 


Tested  Effect 

ANOVA  Result 

HUD 

F(3,33)  =  4.46;  p  =  .010 

Approach  Segment 

F(l,ll)  =  3.51;  p  =  .088 

HUD  X  Approach  Segment 

F(3,33)  =  2.16;  p  =  .lll 

The  significant  main  effect  of  HUD  on  workload  was  further  investigated  with  a  post- 
hoc  Duncan  test,  which  showed  that  workload  ratings  for  the  Full  HUD  were  significantly 
larger  than  those  for  the  Basic/BOTH  Symbology  Set  but  did  not  differ  from  those  for  the 
Basic/RWY  and  Basic/RAW  Sets. 

The  means  used  in  this  analysis  are  provided  in  Table  B-27. 


Table  B-27.  Means  Used  in  the  Workload  Analysis 


Basic/RWY 

Basic/RAW 

Basic/BOTH 

Full 

Mean 

0.1458 

0.0933 

0.0840 

0.1768 
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APPENDIX  C  -  SUMMARY  OF  QUESTIONNAIRE  RESPONSES 
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Introduction 


The  questionnaire  used  in  the  Head-Up  Display  (HUD)  Symbology  Evaluation  for  the 
Autonomous  Landing  Guidance  (ALG)  System  is  provided  in  Appendix  A.  The  responses  of 
the  12  pilots  who  participated  in  this  study  are  summarized  in  this  Appendix.  Where 
appropriate,  the  responses  are  summarized  in  tables.  Comments  provided  by  the  pilots  have 
also  been  recorded. 


Question  1.  Full  HUD  Symbology  Set 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the 
tables  below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the  Full 
HUD  Symbology  Set  in  the  performance  of  each  of  these  tasks  for  each  of  the  out-the- 
window  (OTW)  conditions  identified.  Please  provide  comments  and  suggested  design 
alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E). 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor 
issues  having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation ; 
changes  would  be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations 
impair  pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to 
mission  failure;  redesign  required. 


1.  deforethe  eeneor  itnag e/visual  scene  was  available: 


a)  Safely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

Table  C-1  represents  the  frequency  with  which  each  of  the  above  Acceptability  Ratings 
were  used  to  respond  to  the  above  matrix  in  regards  to  the  Full  HUD  Symbology  Set. 
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Table  C-1.  Frequency  of  Acceptability  Rating  Responses  to  Question  1  Regarding  the  Full 

HUD  Symbology  Set 


Rating 

la.  With  Sensor 

lb.  With  Sensor 

Ic.  With  Sensor 

Id.  With  Sensor 

le.  With  Sensor 

If.  With  Sensor 

la.  Corner  Reflectors 

lb.  Corner  Reflectors 

Ic.  Corner  Reflectors 

Id,  Corner  Reflectors 

le.  Corner  Reflectors 

jlf.  Corner  Reflectors 

la.  No  Sensor 

Ib.  No  Sensor 

Ic.  No  Sensor 

|ld.  No  Sensor 

le.  No  Sensor 

k. 

o 

s 

CZ3 

o 

Z 

Full  HUD 

A 

pn 

3 

3 

B 

5 

4 

2 

3 

3 

2 

5 

4 

2 

3 

3 

2 

B 

4 

4 

4 

5 

4 

5 

5 

6 

8 

6 

1 

4 

3 

5 

6 

3 

C 

4 

5 

5 

3 

1 

2 

4 

5 

4 

3 

1 

1 

5 

4 

4 

3 

3 

3 

D 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

1 

0 

2 

2 

2 

1 

1 

1 

E 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Comments: 

Pilot  1 :  Strongly  suggest  fitting  a  contrast  feature  to  clearly  define  the  differences 

between  the  projected  HUD  information  which  is  green,  and  the  bright  white 
day  time  cloud  background.  With  the  Full  HUD  data  display  selected,  it 
was  sometimes  impossible  to  make  out  the  different  HUD  depiction’s, 
especially  the  vertical  course  references. 

Pilot  2:  Very  cluttered  display  that  gets  more  confusing  with  increasing  errors.  “No 
Sensor”  has  very  poor  contrast  and  hard  to  see  overall. 

Pilots:  Remove  some  of  the  clutter.  Suggestions:  “VanHalen”  (aircraft  reference) 
symbol,  speed-worm.  At  times  I  had  difficulty  discerning  my  glideslope 
and  course  information  (critical  information)  from  other  not  so  critical 
information. 

Pilot  7:  The  Full  HUD  contained  too  much  information  and  was  overwhelming.  The 
following  symbols  need  to  be  removed:  AIRCRAFT  nose  pointer;  vertical 
dot  scale  (see  drawing);  tail  yaw  indicator;  flare  indicator;  runway  rise 
indicator;  runway  marker  (only  until  short  final  -  e.g.,  2  nm);  acceleration 
cue  (not  needed  when  beginning  landing  transition).  Additionally,  I  would 
like  to  see  the  flight  path  marker  and  pitch  scale  look  something  like  this: 
(see  drawing).  The  acceleration  cue  and  airspeed  bar  are  nice  to  have 
indications,  but  the  scale  is  too  small  (i.e.,  small  deviations  result  in  large 
marked  displacement). 

Pilot  11:  Display  in  General/Symbols  Used:  Miniature  AIRCRAFT...pitch  symbol  is 
too  large,  needs  to  be  smaller  and  a  different  shape,  the  U.S.  Mil  symbol 
used  on  ADIs  would  be  fine;  Flight  Path  Marker... unfamiliar  shape.  The 
“legs”  don’t  do  anything  except  identify  the  symbol  as  the  FPM.  Change 
symbol  to  (see  drawing),  wings  and  tail  can  be  used  to  overlay  other 
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symbols;  Digital  Readouts  —  make  more  work  than  analog  pointers;  Digital 
VVI...“sucks.” 


Pilot  12:  Before  sensor  image/visual  acquisition,  all  modes  were  identical  in  the  “full 
HUD”  design.  The  Aircraft  symbol/lubber  line  was  distracting  and  should 
be  eliminated. 
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Question  1 .  Basic/RWY  HUD  Symbology  Set 

Instructions;  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the 
tables  below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the 
Basic/RWY  HUD  Symbology  Set  in  the  performance  of  each  of  these  tasks  for  each  of 
the  out-the-window  (OTW)  conditions  identified.  Please  provide  comments  and 
suggested  design  alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E). 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor 
issues  having  no  impact  on  pilot  performance. 

C.  Borderline;  Incomplete  information  and/or  troublesome  interpretation  ; 
changes  would  be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations 
impair  pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to 
mission  failure;  redesign  required. 


1.  before  the  eeneor  ima^e/visual  scene  was  available: 


a)  Safely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

— - - - -  — 

f)  Determine  position  alon^  the  approach  path 

Table  C-2  represents  the  frequency  with  which  each  of  the  above  Acceptability  Ratings 
were  used  to  respond  to  the  matrix  in  regards  to  the  Basic/RWY  HUD  Symbology  Set. 
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Table  C-2.  Frequency  of  Acceptability  Rating  Responses  to  Question  1  Regarding  the 

Basic/RWYHUD  Symbology  Set 


Rating 

la.  With  Sensor 

lb.  With  Sensor 

Ic.  With  Sensor 

Id.  With  Sensor 

le.  With  Sensor 

If.  With  Sensor 

la.  Corner  Reflectors 

lb.  Corner  Reflectors 

Ic.  Corner  Reflectors 

Id.  Corner  Reflectors 

le.  Corner  Reflectors 

If  Corner  Reflectors 

la.  No  Sensor 

lb.  No  Sensor 

Ic.  No  Sensor 

Id.  No  Sensor 

le.  No  Sensor 

If  No  Sensor 

Basic/RWY 

A 

3 

1 

1 

1 

2 

3 

1 

2 

1 

2 

1 

2 

1 

0 

1 

1 

0 

B 

3 

5 

4 

9 

6 

3 

5 

3 

8 

5 

6 

1 

4 

3 

6 

6 

6 

C 

2 

4 

2 

1 

3 

2 

3 

3 

3 

2 

3 

2 

5 

3 

3 

3 

2 

2 

D 

4 

2 

4 

1 

1 

3 

3 

4 

1 

2 

3 

4 

3 

6 

2 

3 

4 

E 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

Comments: 

Pilot  1 :  Much  easier  to  work  with  than  the  Full  HUD.  The  flight  path  indicator 

makes  it  a  lot  easier  to  anticipate  the  actual  aircraft  flight  path,  and  to  put  in 
corrections  ahead  of  time  instead  of  reacting  to  changed  aircraft  conditions. 

Pilot  2:  See  my  general  comments. 

Pilot  3:  Basic/RWY  is  near  impossible  to  attain  centerline  more  than  1  mile  out. 

Pilot  5:  Runway  symbol  does  provide  crude  course  and  glideslope  information,  but 
is  not  sufficient  for  flying  a  precision  approach  in  low  visibility  conditions. 

I  landed  short  at  least  once  with  this  display. 

Pilot  6:  Cues  very  subtle!  Also,  lack  of  DME  hurts  the  determination  of  position 
along  the  approach  path. 

Pilot  7:  The  lack  of  a  definitive  glideslope  cue  really  hampered  my  ability  to 
visualize/determine  my  position  along  the  glidepath.  The  Runway 
perspective  (long  narrow  lines)  made  me  feel  that  I  was  consistently  high 
on  glideslope.  The  gull  wing  FPM  allowed  for  some  interpretation  as  to 
where  the  “center”  actually  was. 

Pilot  1 1 ;  Symbol  comments  same  as  before.  In  general,  I  learned  to  use  the 

minimum  symbol  set... whatever  was  easiest  for  me.. .seldom  used  other 
symbology.  Background  (VFR,  sensor  reflectors,  haze)  didn’t  matter.  This 
display  was  the  hardest  to  use.  Okay  if  on  course  and  glideslope,  but 
harder  to  interpret  if  high  or  low,  left  or  right.  Would  make  corrections 
then  wait  to  see  what  happened. 
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Question  1-  The  Rasic/RAW  HUD  Svmbologv  Set 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the 
tables  below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the 
Basic/RAW  HUD  Symbology  Set  in  the  performance  of  each  of  these  tasks  for  each  of 
the  out-the-window  (OTW)  conditions  identified.  Please  provide  comments  and 
suggested  design  alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E). 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor 
issues  having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation  ; 
changes  would  be  desirable. 

D.  Moderately  imacceptable:  Lack  of  information  and/or  difficult  interpretations 
impair  pilot  performance;  corrections  required. 

E.  Very  imacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to 
mission  failure;  redesign  required. 


Safely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

Table  C-3  represents  the  frequency  with  which  each  of  the  above  Acceptability  Ratings 
were  used  to  respond  to  the  matrix  in  regards  to  the  Basic/RAW  HUD  Symbology  Set. 
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Table  C-3.  Frequency  of  Acceptability  Rating  Responses  to  Question  1  Regarding  the 

Basic/RA  W  HUD  Symbology  Set 


jRating 

la.  With  Sensor 

lb.  With  Sensor 

Ic.  With  Sensor  | 

Id.  With  Sensor 

le.  With  Sensor 

If.  With  Sensor 

la.  Corner  Reflectors 

Ib.  Corner  Reflectors 

Ic.  Corner  Reflectors  | 

Id.  Corner  Reflectors 

le.  Corner  Reflectors 

If.  Corner  Reflectors 

la.  No  Sensor 

lb.  No  Sensor 

Ic.  No  Sensor 

Id.  No  Sensor  | 

le.  No  Sensor  | 

If.  No  Sensor 

Basic/RAW 

A 

5 

5 

5 

6 

5 

4 

5 

6 

6 

6 

5 

4 

3 

4 

3 

3 

3 

rr- 

B 

5 

4 

5 

3 

5 

6 

5 

3 

4 

3 

5 

6 

4 

5 

5 

5 

7 

8 

C 

2 

2 

1 

3 

1 

1 

2 

2 

1 

3 

1 

1 

3 

1 

2 

2 

0 

0 

D 

0 

1 

1 

0 

1 

1 

0 

1 

1 

0 

1 

1 

2 

2 

2 

2 

2 

2 

E 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Comments: 

Pilot  2:  See  general  comment  page. 

Pilot  6:  Need  DME  data  or  some  other  way  to  determine  distance  from  threshold. 
Contrast  problems  again  in  “No  Sensor”  case. 

Pilot  7:  Overall  easy  to  fly.  A  flight  director  ball  would  have  made  it  a  little  easier. 

Pilot  1 1 :  C  ratings  =  symbol  problems.  Felt  that  I  performed  best  with  this  display, 
easy  to  imderstand,  least  clutter. 
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Question  1.  The  Basic/BOTH  HUD  Symbology  Set 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the 
tables  below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the 
Basic/BOTH  HUD  Symbology  Set  in  the  performance  of  each  of  these  tasks  for  each  of 
the  out-the-window  (OTW)  conditions  identified.  Please  provide  comments  and 
suggested  design  alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E). 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor 
issues  having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation  ; 
changes  would  be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations 
impair  pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to 
mission  failure;  redesign  required. 


1.  3efore  the  eer\eor  ima^e/visual  scene  was  available: 


L_ - - - - - 

a)  Safely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

Table  C-4  represents  the  frequency  with  which  each  of  the  above  Acceptability  Ratings 
were  used  to  respond  to  the  matrix  in  regards  to  the  Basic/BOTH  HUD  Symbology  Set. 
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Table  C-4.  Frequency  of  Acceptability  Rating  Responses  to  Question  1  Regarding  the 

Basic/BOTH  HUD  Symbology  Set 


Rating 

la.  With  Sensor 

lb.  With  Sensor 

Ic.  With  Sensor 

1(1.  With  Sensor 

le.  With  Sensor 

If.  With  Sensor 

la.  Corner  Reflectors 

lb.  Corner  Reflectors 

Ic.  Corner  Reflectors 

Id.  Corner  Reflectors 

le.  Corner  Reflectors 

If.  Corner  Reflectors 

la.  No  Sensor 

lb.  No  Sensor 

Ic.  No  Sensor 

Id.  No  Sensor 

le.  No  Sensor 

If.  No  Sensor 

Basic/BOTH 

A 

5 

4 

5 

6 

5 

4 

6 

5 

6 

7 

6 

4 

3 

3 

3 

5 

4 

3 

B 

7 

7 

6 

5 

6 

7 

6 

6 

5 

4 

5 

7 

6 

7 

6 

4 

6 

7 

C 

0 

1 

1 

1 

1 

1 

0 

1 

1 

1 

1 

1 

2 

1 

2 

2 

1 

1 

D 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 

1 

1 

E 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Comments: 

Pilot  2:  Having  runway  symbol  helps,  but  trying  to  interpret  different  systems  seems 
to  cause  confusion  if  picture  isn’t  “obvious.” 

Pilot  6:  A  good  set  of  information  except  with  contrast  problems  in  “No  Sensor” 
cases.  No  DME...but  less  of  a  factor  due  to  runway  presence. 

Pilot  7:  With  “No  Sensor,”  contrast  is  too  high... function  of  simulation  display. 

Pilot  1 1 :  Symbol  problems.. .same.  Second  easiest  to  fly.  Runway  symbol  provides 
most  information  when  close  in. 
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Question  2.  Full  HUD  Symbology  Set 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the 
tables  below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the  Full 
HUD  Symbology  Set  in  the  performance  of  each  of  these  tasks  for  each  of  the  out-the- 
window  (OTW)  conditions  identified.  Please  provide  comments  and  suggested  design 
alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E). 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor 
issues  having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation  ; 
changes  would  be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations 
impair  pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  xmsafe,  impractical,  and  contributes  greatly  to 
mission  failure;  redesign  required. 


a)  Safely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

Determine  desired  touchdown  point 

Table  C-5  represents  the  frequency  with  which  each  of  the  above  Acceptability  Ratings 
were  used  to  respond  to  the  matrix  in  regards  to  the  Full  HUD  Symbology  Set. 
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Table  C-5.  Frequency  of  Acceptability  Rating  Responses  to  Question  2  Regarding  the  Full 

HUD  Symbology  Set 


Rating 

2a.  VMC 

2b.  VMC 

u 

S 

> 

c 

C3 

2d.  VMC 

2c.  VMC 

2f.  VMC 

2g.  VMC 

2a.  With  Sensor 

2b.  With  Sensor 

2c.  With  Sensor 

2d.  With  Sensor 

2e.  With  Sensor 

2f.  With  Sensor 

2g.  With  Sensor 

2a.  Corner  Reflectors 

2b.  Corner  Reflectors 

2c.  Corner  Reflectors 

2d.  Corner  Reflectors  j 

2e.  Corner  Reflectors 

2f.  Corner  Reflectors 

2g.  Corner  Reflectors  j 

■■ 

B 

B 

B 

B 

6 

7 

7 

5 

5 

5 

4 

5 

7 

6 

3 

5 

3 

J, 

5 

6 

2 

B 

B 

3 

3 

5 

3 

4 

3 

4 

4 

4 

6 

4 

4 

4 

5 

3 

5 

3 

n" 

C 

3 

3 

3 

2 

B 

B 

B 

3 

3 

3 

2 

3 

1 

2 

4 

4 

3 

2 

3 

1 

lL 

D 

0 

0 

0 

0 

D 

B 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

1 

0 

2 

E 

0 

0 

0 

0 

0 

0 

0 

B 

0 

B 

B 

0 

B 

B 

B 

B 

0 

0 

0 

0 

[± 

Comments: 

Pilot  1 :  Overall,  once  I  acquired  a  good  visual  with  the  landing  runway  (i.e.,  runway 
end  identifier  lights,  rabbits,  runway  markings),  I’m  out  of  the  instrument 
mode,  and  about  the  only  thing  I’ll  reference  is  the  airspeed.  The  rest  is  all 
visual  cues. 

Pilot  2;  Clutter  primary  problem... even  in  VMC  hard  to  see  runway  environment. 

Becomes  worse  problem  with  comer  reflectors  because  cues  are  more  subtle. 

Pilot  6:  “B”  rating  given  for  clutter  on  Full  HUD  display.  Question  2. ..at  certain 
points  had  to  look  past  HUD  information  to  real  world.  On  question  1,  the 
“No  Sensor”  set  was  most  difficult  due  to  contrast  of  HUD  symbology  with 
gray  background. 

Pilot  7:  Not  necessary  in  VMC.  On  question  2e:  Full  HUD  tends  to  obscure  OTW 
view  of  runway  environment  in  almost  all  cases.  2g:  Same  as  2e.  Only 
applied  to  the  desired  TD  point. 

Pilot  10:  Full  HUD  needs  to  be  decluttered.  “Flag”  doesn’t  seem  appropriate.  Add 
turn  rate/slip  indicator,  i.e.,  needle  and  ball.  Suggest  orange  color  for  “no 
sensor”  IMC  for  contrast.  Full  HUD  interferes  with  transition  to  visual 
during  landing  with  no  sensor. 

Pilot  1 1 :  Display  in  General/Symbols  Used:  Miniature  AIRCRAFT... pitch  symbol  is 
too  large,  needs  to  be  smaller  and  a  different  shape,  the  U.S.  Mil  symbol  used 
on  ADIs  would  be  fine;  Flight  Path  Marker.. .unfamiliar  shape.  The  “legs” 
don’t  do  anything  except  identify  the  symbol  as  the  FPM.  Change  symbol  to 
(see  drawing),  wings  and  tail  can  be  used  to  overlay  other  symbols;  Digital 
Readouts. ..make  more  work  than  analog  pointers;  Digital  VVI... “sucks.” 

Pilot  12:  Comer  reflectors  did  not  provide  enough  depth  perception  for  “comfortable” 
vertical  control . . .  long  landings  or  firm  touchdowns  resulted. 
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Question  2.  The  Basic/RWY  HUD  Symbology  Set 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the 
tables  below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the 
Basic/RWY  HUD  Symbology  Set  in  the  performance  of  each  of  these  tasks  for  each  of 
the  out-the-window  (OTW)  conditions  identified.  Please  provide  comments  and 
suggested  design  alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E). 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor 
issues  having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation  ; 
changes  would  be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations 
impair  pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to 
mission  failure;  redesign  required. 


2.  After  t^ie  seneor  \maqeJv\eua\  scene  was  available: 


a)  Safely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

q)  Determine  desired  touchdown  point 

Table  C-6  represents  the  frequency  with  which  each  of  the  above  Acceptability  Ratings 
were  used  to  respond  to  the  matrix  in  regards  to  the  Basic/RWY  HUD  Symbology  Set. 
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Table  C-6.  Frequency  of  Acceptability  Rating  Responses  to  Question  2  Regarding  the 

Basic/RWY HUD  Symbology  Set 


Rating 

2a.  VMC 

2b.  VMC 

o 

S 

> 

u 

u 

S 

> 

2e.  VMC 

2f.  VMC 

2g.  VMC 

2a.  With  Sensor 

2b.  With  Sensor  | 

2c.  With  Sensor 

j2d.  With  Sensor 

|2c.  With  Sensor 

2f.  With  Sensor  j 

im 

O 

S 

CJ 

tn 

"B 

1 

ei) 

j2a.  Corner  Reflectors 

2b.  Corner  Reflectors 

2c,  Corner  Reflectors 

2(1.  Corner  Reflectors 

j2e.  Corner  Reflectors 

2f.  Corner  Reflectors 

2g.  Corner  Reflectors 

Basic/RWY 

A 

6 

6 

7 

6 

5 

7 

7 

6 

3 

4 

5 

LH 

5 

4 

5 

2 

B 

B 

4 

4 

B 

3 

3 

3 

3 

3 

4 

3 

2 

2 

6 

3 

5 

4 

3 

2 

B 

B 

B 

4 

i 

C 

2 

1 

2 

1 

2 

2 

1 

2 

3 

1 

3 

3 

2 

4 

4 

4 

1 

3 

3 

6 

D 

1 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

E 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

_0_ 

0 

0 

0 

0 

0 

0 

Comments: 

Pilot  1 :  As  long  as  you’ve  got  the  correct  manually-selected  glideslope  (the  dashed 
line)  indicator  depicted,  it’s  easy. 

Pilot  2:  Fairly  good  symbology.  Could  be  enhanced  with  course,  DME,  vwnds,  etc. 
Runway  symbol  accuracy  not  the  greatest  but  acceptable. 

Pilot  6:  Symbology  was  ignored  once  visual  was  sighted  until  flare  was  used  to  a 
mild  extent  in  the  comer  reflector  scenario  especially  as  a  flare  cue. 

Pilot  7:  2a  in  VMC... runway  lines  didn’t  quite  interfere  with  visual,  but  I  could  have 
done  without  them.  (Not  necessary).  2a  with  Sensor  was  excellent!  2a  with 
comer  reflectors  tended  to  overlay  and  obscure  comer  reflectors.  2c.. .vertical 
deviation  tough  to  judge.  2f  ..See  above. 

Pilot  9:  The  runway  “captain’s  bars”  (1,000  foot  markers  on  the  runway  symbology) 
are  not  in  sight  until  short  final.  This  makes  it  hard  to  determine  where  you 
are  on  the  glidepath  when  further  out  on  the  approach. 

Pilot  1 1 :  Symbol  comments  same  as  before.  In  general,  I  learned  to  use  the  minimum 
symbol  set.. .whatever  was  easiest  for  me.. .seldom  used  other  symbology. 
Background  (VFR,  sensor  reflectors,  haze)  didn’t  matter.  This  display  was 
the  hardest  to  use.  Okay  if  on  course  and  glideslope,  but  harder  to  interpret  if 
high  or  low,  left  or  right.  Would  make  corrections  then  wait  to  see  what 
happened. 
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Question  2.  The  Basic/RAW  HUD  Svmbologv  Set 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the 
tables  below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the 
Basic/RAW  HUD  Symbology  Set  in  the  performance  of  each  of  these  tasks  for  each  of 
the  out-the-window  (OTW)  conditions  identified.  Please  provide  comments  and 
suggested  design  alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E). 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor 
issues  having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation  ; 
changes  would  be  desirable. 

D.  Moderately  imacceptable:  Lack  of  information  and/or  difficult  interpretations 
impair  pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to 
mission  failure;  redesign  required. 


2.  After  the  sensor  image/visual  scene  was  available: 


- - - — - - 

a)  Safely  fly  the  approach 

b)  Monitor  and  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  ansi  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon0  the  approach  path 

0)  Determine  desired  touchdown  point 

Table  C-7  represents  the  frequency  with  which  each  of  the  above  Acceptability  Ratings 
were  used  to  respond  to  the  matrix  in  regards  to  the  Basic/RAW  HUD  Symbology  Set. 


C-16 


102 


Table  C-7.  Frequency  of  Acceptability  Rating  Responses  to  Question  2  Regarding  the 

Basic/RA  W  HUD  Symbology  Set 


Rating 

u 

> 

K 

-N_ 

2b.  VMC 

2c.  VMC 

2d.  VMC 

> 

O  . 

y 

s 

> 

y 

> 

m£imm 

2a.  With  Sensor 

2b.  With  Sensor 

2c.  With  Sensor 

2d.  With  Sensor 

2e.  With  Sensor 

2f.  With  Sensor 

2g.  With  Sensor 

2a.  Corner  Reflectors 

2b.  Corner  Reflectors 

2c.  Corner  Reflectors 

2d.  Corner  Reflectors 

2c.  Corner  Reflectors 

2f.  Corner  Reflectors 

2g.  Corner  Reflectors 

Basic/RA  W 

A 

8 

8 

8 

7 

6 

9 

7 

8 

5 

7 

7 

7 

8 

5 

6 

5 

6 

6 

5 

4 

B 

3 

3 

3 

3 

3  , 

4 

2 

4 

3 

6 

4 

5 

4 

4 

5 

5 

4 

5 

5 

6 

5 

C 

1 

1 

1 

2 

1  1 

2 

1 

1 

n“ 

1 

1 

0 

1 

0 

H 

n 

H 

n 

1 

1 

3 

D 

0 

O 

0 

D 

0 

0 

D 

11 

0 

0 

0 

0 

0 

0 

0 

0 

D 

0 

0 

0 

0 

E 

0 

0 

0 

0 

0  : 

0 

0 

Ll 

0 

0 

0 

0 

0 

0 

0 

[I 

0 

0 

0 

0 

Comments: 

Pilot  1 :  Altitude  warning  indications  would  be  nice;  overall  the  best  of  the  three 
systems  to  use. 

Pilot  6:  Once  visual... basically  okay. 

Pilot  7:  Don’t  need  it  in  VMC.  Very  helpful  with  sensor  video;  it  really  worked 
well  to  determine  lateral  deviation  and  touchdown  point. 

Pilot  1 1 :  C  ratings  =  symbol  problems.  Felt  that  I  performed  best  with  this  display, 
easy  to  understand,  least  clutter. 
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Question  2.  The  Basic/BOTH  HUD  Svmbologv  Set 

Instructions:  The  ALG  precision  approach  is  segmented  into  a  set  of  unique  tasks  in  the 
tables  below.  Use  the  Acceptability  Scale  provided  to  rate  the  suitability  of  the 
Basic/BOTH  HUD  Symbology  Set  in  the  performance  of  each  of  these  tasks  for  each  of 
the  out-the-window  (OTW)  conditions  identified.  Please  provide  comments  and 
suggested  design  alternatives  for  any  item  rated  “borderline  or  worse”  (i.e.,  C,  D,  or  E). 


Acceptability  Scale 

A.  Very  acceptable:  All  information  readily  available  and  easily  interpreted. 

B.  Moderately  acceptable:  All  information  available  and  interpretable;  minor 
issues  having  no  impact  on  pilot  performance. 

C.  Borderline:  Incomplete  information  and/or  troublesome  interpretation ; 
changes  would  be  desirable. 

D.  Moderately  unacceptable:  Lack  of  information  and/or  difficult  interpretations 
impair  pilot  performance;  corrections  required. 

E.  Very  unacceptable:  Display  is  unsafe,  impractical,  and  contributes  greatly  to 
mission  failure;  redesign  required. 


a)  Safely  fly  the  approach 

b)  Monitor  an^i  control  lateral  deviation 

c)  Monitor  and  control  vertical  deviation 

d)  Monitor  and  control  fundamental  aircraft  performance 

e)  Monitor  and  control  ground  track 

f)  Determine  position  alon^  the  approach  path 

3)  Determine  desired  touchdown  point 

Table  C-8  represents  the  frequency  with  which  each  of  the  above  Acceptability  Ratings 
were  used  to  respond  to  the  matrix  in  regards  to  the  Basic/BOTH  HUD  Symbology  Set. 
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Table  C-8.  Frequency  of  Acceptability  Rating  Responses  to  Question  2  Regarding  the 

Basic/BOTH  HUD  Symbology  Set 


bt) 

.£ 

u 

S 

> 

2b.  VMC 

2c.  VMC 

2d.  VMC 

u 

> 

2f.  VMC 

2g.  VMC 

2a.  With  Sensor 

2b.  With  Sensor 

2c.  With  Sensor 

2d.  With  Sensor 

be.  With  Sensor 

2f.  With  Sensor 

2g.  With  Sensor 

2a.  Corner  Reflectors 

2b.  Corner  Reflectors 

2c,  Corner  Reflectors 

2d.  Corner  Reflectors 

2e.  Corner  Reflectors 

2f.  Corner  Reflectors 

2g.  Corner  Reflectors 

Basic/BOTH 

A 

8 

9 

8 

7 

9 

7 

7 

7 

7 

8 

7 

7 

6 

6 

B 

B 

B 

D 

B 

B 

2 

2 

1 

2 

H 

3 

2 

5 

4 

4 

4 

4 

4 

5 

4 

5 

3 

5 

3 

6 

4 

C 

1 

2 

2 

2 

1 

2 

1 

0 

1 

1 

1 

0 

1 

0 

2 

n 

B 

n 

n 

2 

3 

D 

O 

0 

0 

0 

0 

O 

O 

a 

0 

0 

0 

0 

0 

0 

0 

o 

D 

D 

D 

0 

0 

E 

Z] 

0 

0 

0 

0 

z 

z 

z 

0 

0 

0 

0 

0 

0 

o 

m 

Z 

Z 

z 

0 

0 

Comments: 

Pilot  1 :  Essentially,  the  same  as  the  basic  HUD  with  raw  deviation  data  only.  The 
difference  between  having  the  runway  outline  depiction,  and  just  paying 
attention  to  the  radar  altimeter  is  slight,  and  not  worth  the  clutter;  especially 
when  you’re  4  or  5  miles  or  more  out  and  the  dotted  course  line,  the  raw  data 
line  and  the  runway  all  blend  together. 

Pilot  2:  Symbols  clutter  visual  field. 

Pilot?:  Not  necessary  in  VMC.  All  2s  with  Sensor  outstanding!  The  only  thing  I 
would  change  from  what  you  have  is:  1)  take  away  runway  symbol  until 
short  (2nm)  final;  2)  provide  flight  director  ball;  and  3)  add  acceleration  cue 
(imtil  flare)  and  A/S  tape.  2f  and  2g  with  Comer  Reflectors... tough  to 
determine  3-D  position  of  touchdown  point  in  close. 

Pilot  IT.  Symbols.. .same.  Second  easiest  to  fly.  Runway  symbol  provides  most 
information  when  close  in. 
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Question  3.  Use  the  above  Acceptability  Scale  to  rate  the  overall  acceptability  of  each  of  the 
HUD  Symbology  Sets. _ 


Table  C-9.  Frequency  of  Acceptability  Ratings  for  Question  3 


Full  HUD 

Basic/RWY 

Basic/RAW 

Basic/BOTH 

A 

3 

1 

4 

4 

B 

4 

4  1 

6 

5 

c 

5 

5 

1 

2 

D 

0 

2 

1 

1 

E 

0 

0 

0 

0 

Comments: 

Full  HUD  Symbology  Set 

Pilot  12:  Remove  aircraft  symbology. 
The  Basic/RWY  HUD  Symbology  Set 
No  comments  were  provided. 

The  Basic/RAW  HUD  Symbology  Set 
No  comments  were  provided. 

The  Basic/BOTH  HUD  Symbology  Set 
No  comments  were  provided. 
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Question  4.  Use  the  Modified  Cooper  Harper  Scale  (last  page  of  this  package)  to  rate  the 
workload  associated  with  the  Full  HUD  in  each  of  the  following  conditions: 

a.  VMC _ 

Full  HUD  Symbology  Set 

Table  C-1 0.  Frequency  of  Cooper-Harper  Ratings  to  Question  4a  for  the  Full  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

2 

2 

4 

3 

3 

4 

2 

5 

1 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 


The  Basic/RWY  HUD  Symbology  Set 

Table  C-1 1.  Frequency  of  Cooper-Harper  Ratings  to  Question  4a  Basic/RWY  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

2 

2 

4 

3 

4 

4 

2 

5 

0 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 


C-21 


1  07 


The  Basic/RAW  HUD  Symbology  Set 


Table  C-12.  Frequency  of  Cooper-Harper  Ratings  to  Question  4a  for  the  Basic/RA  W  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

3 

2 

6 

3 

2 

4 

0 

5 

1 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

Pilot  7:  Not  too  annoying,  but  unnecessary. 


The  Basic/BOTH  HUD  Symbology  Set 

Table  C-13.  Frequency  of  Cooper-Harper  Ratings  to  Question  4a  for  the  Basic/BOTH  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

5 

2 

3 

3 

3 

4 

0 

5 

1 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

Pilot?:  Don’t  need  it. 
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Question  4.  Use  the  Modified  Cooper  Harper  Scale  (last  page  of  this  package)  to  rate  the 
workload  associated  with  the  Full  HUD  in  each  of  the  following  conditions: 

b.  With  a  sensor  image _ 

Full  HUD  Symbology  Set 

Table  C-14.  Frequency  of  Cooper-Harper  Ratings  to  Question  4b  for  the  Full  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

1 

2 

2 

3 

3 

4 

3 

5 

n 

0 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 


The  Basic/RWY  HUD  Symbology  Set 

Table  C-15.  Frequency  of  Cooper-Harper  Ratings  to  Question  4b  for  the  Basic/RWY  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

0 

2 

2 

3 

2 

4 

5 

5 

1 

6 

1 

7 

1 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 
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The  Basic/RAW  HUD  Symbology  Set 


Table  C-16.  Frequency  of  Cooper-Harper  Ratings  to  Question  4b  for  the  Basic/RAW  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

1 

2 

4 

3 

5 

4 

1 

5 

1 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

Pilot  7:  Would  like  to  see  flight  director. 


The  Basic/BOTH  HUD  Symbology  Set 

Table  C-1 7.  Frequency  of  Cooper-Harper  Ratings  to  Question  4b  for  the  Basic/BOTH  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

2 

2 

1 

3 

6 

4 

0 

5 

3 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 
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Question  4.  Use  the  Modified  Cooper  Harper  Scale  (last  page  of  this  package)  to  rate  the 
workload  associated  with  the  Full  HUD  in  each  of  the  following  conditions: 

c.  With  the  comer  reflectors _ 

Full  HUD  Symbology  Set 

Table  C-18.  Frequency  of  Cooper-Harper  Ratings  to  Question  4c  for  the  Full  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

1 

2 

1 

3 

1 

4 

6 

5 

2 

6 

0 

7 

1 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 


The  Basic/RWY  HUD  Symbology  Set 

Table  C-19.  Frequency  of  Cooper-Harper  Ratings  to  Question  4c  for  the  Basic/RWY  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

0 

2 

2 

3 

1 

4 

4 

5 

3 

6 

2 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 
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Pilot  7:  Not  too  annoying,  but  unnecessary.  Would  like  to  see  flight  director. 
Touchdown  aim  point  is  ambiguous. 


The  Basic/RAW  HUD  Symbology  Set 

Table  C-20.  Frequency  of  Cooper-Harper  Ratings  to  Question  4c  for  the  Basic/RAW  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

1 

2 

4 

3 

2 

4 

0 

5 

5 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 


The  Basic/BOTH  HUD  Symbology  Set 

Table  C-21.  Frequency  of  Cooper-Harper  Ratings  to  Question  4c  for  the  Basic/BOTH  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

1 

2 

3 

3 

3 

4 

0 

5 

4 

6 

1 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 
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Question  4.  Use  the  Modified  Cooper  Harper  Scale  (last  page  of  this  package)  to  rate  the 
workload  associated  with  the  Full  HUD  in  each  of  the  following  conditions: 

d.  Without  a  sensor  image _ 

Full  HUD  Symbology  Set 

Table  C-22.  Frequency  of  Cooper-Harper  Ratings  to  Question  4d  for  the  Full  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

1 

2 

0 

3 

4 

4 

2 

5 

1 

6 

3 

7 

0 

8 

1 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 


The  Basic/RWY  HUD  Symbology  Set 

Table  C-23.  Frequency  of  Cooper-Harper  Ratings  to  Question  4d for  the  Basic/RWY  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

0 

2 

1 

3 

2 

4 

3 

5 

2 

6 

1 

7 

2 

8 

1 

9 

0 

10 

0 
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Comments: 

Pilot  7:  Without  glidepath  information  in  the  initial  stages  of  the  approach, 

excessive  mental  effort  is  required  to  maintain  glidepath.  Once  in  close 
with  the  sensor,  it  was  excellent  and  pretty  good  with  the  comer 
reflectors.  VMC  it  was  slightly  annoying  but  tolerable.  Overall,  it  is  an 
okay  system  1)  if  used  with  additional  glidepath.  information  and  2)  if 
runway  lines  are  selectable  by  the  pilot.  A  fixed  value  could  cause  high 
or  low  flares  depending  on  the  runway  width. 


The  Basic/RAW  HUD  Symbology  Set 

Table  C-24.  Frequency  of  Cooper-Harper  Ratings  to  Question  4dfor  the  Basic/RAW  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

1 

2 

2 

3 

2 

4 

2 

5 

2 

6 

1 

7 

1 

8 

1 

9 

0 

10 

0 

Comments: 

Pilot  7:  Not  too  annoying,  but  unnecessary.  Would  like  to  see  flight  director. 
Touchdown  point  is  ambiguous. 
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The  Basic/BOTH  HUD  Symbology  Set 


Table  C-24.  Frequency  of  Cooper-Harper  Ratings  to  Question  4d for  the  Basic/BOTH  HUD 

Symbology  Set 


Cooper-Harper 

Rating 

Frequency 

1 

1 

2 

0 

3 

4 

4 

j 

5 

4 

6 

0 

7 

0 

8 

0 

9 

0 

10 

0 

Comments: 

No  comments  were  provided. 

Question  5.  Did  you  experience  any  clutter  problems  with  the  HUD  Symbology  Sets: 
a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 

Full  HUD  Symbology  Set 
Yes  11  No  1 
Comments: 

Pilot  1 :  In  as  much  as  I  took  time  to  wonder  if  there  was  something  I 

should  be  paying  attention  to  rather  than  concentrating  on  landing 
the  plane. 

Pilot  2:  Obstructed  view  of  image. 

Pilot  3:  Too  much  information,  led  to  stagnated  Cross-check. 

Pilot  4:  Aircraft  symbol  masks  other  data/symbology. 

Pilot  6:  So  much  data  presented,  at  some  instances  (flare)  it  actually  was  a 
distraction. 

Pilot  7:  Too  much  information  tends  to  wash  out  runway  environment. 
Pilot  8:  Too  much  symbology  in  center. 

Pilot  9:  Symbols  sometimes  overlapped,  blocking  important  data. 
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Pilot  10:  Full  HUD  is  very  cluttered. 

Pilot  1 1 :  Confuse  some  symbols,  i.e.,  A/S  worm  with  runway  minimum 
aircraft  blocks  out  other  symbols. 

Pilot  12:  Aircraft  Symbol/Flashing  “GND,”  and  “F”  Flare 


The  Basic/RWY  Symbology  Set 
Yes  4  No  8 
Comments: 

Pilot  2:  Miniature  aircraft  too  large. 

Pilot  4:  Aircraft  symbol  clutters  HUD. 

Pilot  7:  Slightly  annoying  in  VMC.  Washed  out  comer  reflectors. 
Pilot  12:  Aircraft  Symbol/Flashing  “GND,”  and  “F”  Flare 


The  Basic/RAW  Symbology  Set 
Yes  4  No  8 
Comments: 

Pilot  2:  Too  much  stuff  in  central  field. 

Pilot  4:  Aircraft  symbol  clutters  HUD. 

Pilot  7 :  Aircraft  pointer  is  unnecessary. 

Pilot  12:  Aircraft  Symbol/Flashing  “GND,”  and  “F”  Flare 

The  Basic/BOTH  Symbology  Set 

Yes  4  No  8 
Comments: 

Pilot  2:  See  general  comment  sheet. 

Pilot  4:  Aircraft  symbol  clutters  HUD. 

Pilot  5:  I  like  the  runway  symbol  as  you  get  on  short  final  (i.e.,  inside  1/2 
mile).  It  is  most  useful  here  for  aligning  the  aircraft  with  the 
runway. 

Pilot  7:  Standard  (aircraft  nose  pointer  not  necessary). 

Pilot  12:  Aircraft  Symbol/Flashing  “GND,”  and  “F”  Flare 
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Question  5.  Did  you  experience  any  clutter  problems  with  the  HUD  Symbology  Sets: 

b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 

Full  HUD  Symbology  Set 

Yes  11  No  1 
Comments: 

Pilot  1 :  In  my  opinion,  there  was  much  more  information  depicted  than 

necessary  to  safely  fly  the  approach  and  land  the  plane.  Get  rid  of 
the  course  steering  cue  circle. 

Pilot  2:  Hard  to  discern  small  deviations. 

Pilot  3:  Too  much  information,  led  to  stagnated  cross-check. 

Pilot  4:  Aircraft  symbol  masks  other  data/symbology. 

Pilot  5:  “VanHalen”  symbol  was  sometimes  distracting.  Runway  symbol 
would  sometimes  be  superimposed  on  the  raw  data  localizer 
course.  Caused  confusion. 

Pilot  6:  Speed  worm  and  acceleration  cue  a  distraction. 

Pilot  7:  High  mental  effort  required  to  interpret  all  of  the  information 
presented. 

Pilot  9:  Symbols  sometimes  overlapped,  blocking  important  data. 

Pilot  10:  Full  HUD  is  very  cluttered. 

Pilot  1 1 :  Confuse  some  symbols,  i.e.,  airspeed  worm  with  runway 
minimum  aircraft  blocks  out  other  symbols. 

Pilot  12:  Aircraft  Symbol/Flashing  “GND,”  and  “F”  Flare 

The  Basic/RWY  Symbology  Set 

Yes  4  No  8 
Comments: 

Pilot  2:  Miniature  aircraft  too  large. 

Pilot  4:  Aircraft  symbol  clutters  HUD. 

Pilot  5:  Sometimes  confused  runway  symbol  with  localizer  course  raw 
data. 

Pilot  12:  Aircraft  Symbol/Flashing  “GND,”  and  “F”  Flare 
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The  Basic/RAW  Symbology  Set 

Yes  4  No  8 
Comments: 

Pilot  2:  Miniature  AIRCRAFT  and  course  interfere  with  raw  data. 

Pilot  4:  Aircraft  symbol  clutters  HUD. 

Pilot  7:  Aircraft  pointer  is  unnecessary. 

Pilot  12:  Aircraft  Symbol/Flashing  “GND,”  and  “F”  Flare 

The  Basic/BOTH  Symbology  Set 
Yes  7  No  5 
Comments: 

Pilot  1 :  More  than  5  miles  out,  the  dotted  course  line,  raw  data  line,  the 
heading  marker,  and  the  runway  outline  depiction  all  melded 
together. 

Pilot  4:  Aircraft  symbol  clutters  HUD. 

Pilot  5 :  Confused  CDI  with  runway  at  times. 

Pilot  7 :  Standard  (aircraft  nose  pointer  not  necessary). 

Pilot  8 :  Runway  symbol  was  tall  and  skinny  and  could  easily  be  confused 
with  raw  data. 

Pilot  12:  Aircraft  Symbol/Flashing  “GND,”  and  “F”  Flare 

Question  6.  Was  there  any  necessary  information  missing  from  the  any  of  the  HUD 
Symbology  Sets: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 

Full  HUD  Symbology  Set 

Yes  2  No  10 
Comments: 

Pilot  7:  No  FAF  marker  (or  MM)  information  was  presented.  Flight 
director  only  provided  vertical  information;  it  should  do  both 
vertical  and  horizontal. 

Pilot  10:  Tum/slip. 
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The  Basic/RWY  Symbology  Set 

Yes  8  No  4 
Comments: 

Pilot  2:  Course  indicator. 

Pilot  4:  DME. 

Pilot  6:  Course  glideslope. 

Pilot  7:  See  previous  comments. 

Pilot  8:  Raw  data  would  help. 

Pilot  9:  I  would  not  want  to  fly  a  low-visibility  approach  without  raw 
data. 

Pilot  10:  Wind  vector  and  DME. 

Pilot  1 1 :  Maybe  not...wasn’t  sure  how  to  use  it. 

Pilot  12:  See  videotape. 

The  Basic/RAW  Symbology  Set 
Yes  5  No  7 
Comments: 

Pilot  2:  DME  winds,  etc.  Would  be  useful. 

Pilot  3 :  Runway  would  have  helped. 

Pilot  4:  DME. 

Pilot  7:  Not  “necessary”  but  flight  director  ball  is  desirable. 

Pilot  10:  DME  and  wind. 

Pilot  1 2 :  Steer  to  vs.  deviation  from. 

The  Basic/BOTH  Symbology  Set 

Yes  3  No  9 
Comments: 

Pilot  2:  DME,  winds  would  be  helpful. 

Pilot  4:  DME. 

Pilot  10:  Needle/ball,  DME  and  vwnd. 

Pilot  12:  See  videotape. 
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Question  6.  Was  there  any  necessary  information  missing  from  the  any  of  the  HUD 
Symbology  Sets: 

b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 

Full  HUD  Symbology  Set 

Yes  4  No  8 
Comments: 

Pilot  1 :  I  needed  something  to  break  altitude  complacency.  The  radar 

altimeter  and  barometric  altimeter  decision  heights  should  double 
in  size  or  change  color  (green  for  approaching,  yellow  at  altitude 
±  20’,  red  going  below) 

Pilot  4:  DME. 

Pilot  10:  Tum/slip. 

Pilot  12:  Better  lateral  steering  to  localizer . . .  mechanize  pitch  symbol  “o” 
to  ®  or  (see  drawing),  and  have  it  give  steering  to  localizer  and 
glideslope. 

The  Basic/RWY  Symbology  Set 

Yes  10  No  2 
Comments: 

Pilot  2:  Course  indicator  DME  and  winds. 

Pilots:  Location/glideslope. 

Pilot  4:  DME. 

Pilot  5 :  More  precise  course  and  glideslope  deviation  information. 

Pilot  8:  Course  and  glidepath. 

Pilot  9:  I  would  not  want  to  fly  a  low-visibility  approach  without  raw 
data. 

Pilot  10:  Course/glidepath  data. 

Pilot  12:  See  videotape. 

The  Basic/RAW  Symbology  Set 
Yes  6  No  6 


C-34 


120 


Comments: 

Pilot  1 :  Altitude  alerting  indicators.  Reference  ground  speed  depiction. 
Pilot  2:  DME,  winds,  etc.  would  be  useful. 

Pilot  4:  DME. 

Pilot  10:  DME  and  wind. 

Pilot  12;  Steer  to  vs.  deviation  from. 

The  Basic/BOTH  Symbology  Set 

Yes  5  No  7 
Comments: 

Pilot  1 :  Reference  ground  speed.  Altitude  warning  system. 

Pilot  2:  DME,  winds  would  be  helpful. 

Pilot  4;  DME. 

Pilot  6:  DME. 

Pilot  10:  Needle/ball,  DME  and  wind. 

Question  7.  Was  there  any  unnecessary  information  provided  by  the  full  HUD: 

a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 

Full  HUD  Symbology  Set 

Yes  11  No  1 
Comments: 

Pilot  2:  Raw  ILS,  flight  director  cue. 

Pilot  3:  Numerous  approach  guidance  items  were  no  longer  of  use.  Flare, 
GND,  rising  nmway  were  nice,  but  unnecessary. 

Pilot  4:  Aircraft  symbol  and  runway  symbol. 

Pilot  5:  “VanHalen”  and  “Speed- worm.” 

Pilot  8:  Having  reference  mark  “GND”  flash. 

Pilot  9:  I  found  these  symbols  useless  (see  questionnaire  for  drawing). 

Pilot  10:  Flag,  circle,  worm,  carat,  runway,  rising  runway,  GND,  and 
FLARE. 
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Pilot  1 1 :  Speed  worm  and  acceleration  cue  redundant-simulator  hype. 
Pilot  12:  Aircraft  symbol  and  see  videotape. 

The  Basic/RWY  Symbology  Set 

Yes  2  No  10 
Comments: 

Pilot  4:  Aircraft  symbol. 

Pilot  7:  Aircraft  nose  pointer. 

The  Basic/RAW  Symbology  Set 

Yes_4_No_S_ 

Comments: 

Pilot  2:  Min.  AIRCRAFT  symbol. 

Pilot  4:  Aircraft  symbol. 

Pilot  7 :  Aircraft  pointer  is  unnecessary. 

Pilot  8:  No  comment. 

Pilot  9:  The  vertical  dotted  line  of  the  (see  questionnaire  for  drawing) 
symbol. 

Pilot  10:  Raw  data  not  needed  for  landing  phase. 

The  Basic/BOTH  Symbology  Set 

Yes  5  No  7 
Comments: 

Pilot  2:  Using  both  just  clutters  image. 

Pilot  4:  Aircraft  symbol,  runway  symbol. 

Pilot  6:  See  my  HUD  Assembly  Task  selections. 

Pilot  7:  Standard  (aircraft  nose  pointer  not  necessary). 

Pilot  9:  Same  as  before,  the  dotted  vertical  line  and  the  (see 
questionnaire  for  drawing)  symbol. 

Pilot  10:  Runway,  raw  data  and  source  selected. 
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Question  7.  Was  there  any  unnecessary  information  provided  by  the  full  HUD; 

b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 

Full  HUD  Symbology  Set 

Yes  11  No  1 
Comments: 

Pilot  1 :  Get  rid  of  the  acceleration  trend  pointer,  make  the  aircraft 
reference  marker  smaller  and  less  cumbersome. 

Pilot  2:  RAW  ILS,  flight  director  cue. 

Pilot  3:  Airspeed  markers  are  duplicative;  should  have  either/or  far  digital 
or  reel-tape. 

Pilot  4:  Aircraft  s3nnbol  and  runway  symbol. 

Pilot  5:  VanHalen,  speed  worm  and  pointer.,  runway  (at  least  on  long 
final). 

Pilot  9:  I  found  these  symbols  useless  (see  questionnaire  for  drawing). 
Pilot  10:  Flag,  circle,  worm,  carat,  rising  runway  GND,  FLARE. 

Pilot  1 1 :  Speed  worm  and  acceleration  cue  redundant-simulator  hype. 

The  Basic/RWY  Symbology  Set 
Yes  3  No  9 
Comments: 

Pilot  4:  Aircraft  symbol. 

Pilot  7 :  Aircraft  nose  pointer. 

The  Basic/RAW  Symbology  Set 
Yes  4  No  8 
Comments: 

Pilots;  Aircraft  nose  is  not  important. 

Pilot  4:  Aircraft  symbol. 
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The  Basic/BOTH  Symbology  Set 
Yes  7  No  5 
Comments: 

Pilot  2:  Using  both  creates  confusion. 

Pilot  4:  Aircraft  symbol,  runway  symbol. 

Pilot  5:  Runway  symbol  is  not  real  useful  on  long  final. 

Pilot  8:  Runway  symbol. 

Question  8.  Were  there  any  safety  issues  associated  with  any  of  the  HUD  Symbology  Sets: 
a)  When  a  sensor  image/visual  scene  was  available?  Yes _ No _ If  yes,  explain. 

Full  HUD  Symbology  Set 

Yes  6  No  6 
Comments: 

Pilot  2:  Miniature  aircraft  symbol  shouldn’t  be  dominant.  Pitch  scale 
slaved  to  flight  path  marker. 

Pilot  3:  Task  saturation  with  other  events  could  be  hazardous. 

Pilot  6:  Distraction  from  primary  task.. .land  the  airplane. 

Pilot  7:  Only  that  there  is  too  much  information. 

Pilot  9:  Regardless  of  the  clutter  and  symbol  overlapping,  it’s  not  hard  to 
look  past  the  extra  symbols  to  get  the  needed  data. 

Pilot  10:  Clutter  complicates  HUD  interpretation. 

Pilot  1 1 :  Momentarily  confuse  s3Tnbols  after  a  short  distraction,  speed 
worm,  runway  symbol.  Have  to  work  back  to  get  proper 
interpretation. 

The  Basic/RWY  Symbology  Set 
Yes  7  No  5 
Comments: 

Pilot  2:  Lack  of  perspective/runway  centerline  could  be  a  problem  with 
large  course  deviations. 
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Pilot  3;  Centerline  control  was  not  sufficient  for  safe  correcting  if  the  pilot 
wanders. 

Pilot  6:  See  previous  comments  and  HUD  building  exercise. 

Pilot  7:  Lack  of  GP  information. 

Pilot  8:  Not  suitable  for  precision  approaches  without  raw  data  available. 

Pilot  9:  Without  raw  data  and  with  imprecise  glidepath  guidance,  flying  an 
approach  into  mountainous  terrain  could  be  a  problem. 

Pilot  10:  Without  glidepath  deviation  indicator,  it  is  likely  that  a  dragged-in 
approach/short  landing  could  result. 

Pilot  11:1  didn’t  crash. 

The  Basic/RAW  Symbology  Set 
Yes  1  No  11 
Comments: 

Pilot  2:  Raw  data  representation  doesn’t  give  clear  intuitive  picture  of 
required  corrections. 

The  Basic/BOTH  Symbology  Set 
Yes  0  No  12 
Comments: 

No  comments  were  provided. 

Question  8.  Were  there  any  safety  issues  associated  with  any  of  the  HUD  Symbology  Sets: 

b)  When  a  sensor  image/visual  scene  was  not  available?  Yes _ No _ ?  If  yes, 

explain. 

Full  HUD  Symbology  Set 

Yes  5  No  7 
Comments: 

Pilot  2:  Minimize  Aircraft  symbol;  shouldn’t  be  dominant.  Pitch  scale 
slaved  to  flight  path  marker. 

Pilot  3 :  Pilot  workload  and  cross-check  stagnation. 
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Pilot  5 ;  Clutter  was  an  issue,  but  probably  not  to  the  point  that  it  was 
unsafe. 

Pilot?:  Too  much  information. 

Pilot  10:  Clutter  complicates  HUD  interpretation. 

Pilot  1 1 :  Momentarily  confuse  symbols  after  a  short  distraction,  speed 
worm,  runway  symbols.  Have  to  work  back  to  get  proper 
interpretation. 

The  Basic/RWY  Symbology  Set 
Yes  10  No  2 

Comments: 

Pilot  1 :  Contrast  between  the  green  depictions  of  the  HUD  and  the  bright 
white  background  was  a  serious  problem. 

Pilot  2:  Lack  of  perspective/runway  centerline  could  be  a  problem  with 
large  course  deviations. 

Pilot  3:  Excessive  lateral  displacement  as  well  as  vertical  deviation  can 
develop. 

Pilot  5 :  I  flew  two  approaches  to  the  over-run  because  the  glideslope 

information  from  the  nmway  symbol  is  not  sufficiently  precise. 

Pilot  6:  Only  workload  necessary  to  correct  approach  errors. 

Pilot  7 :  Lack  of  GP  information. 

Pilot  10;  Without  glidepath  deviation  indicator,  it  is  likely  that  a  dragged-in 
approach/short  landing  could  result. 

Pilot  1 1 :  Okay,  maybe  I  did. 

The  Basic/RAW  Symbology  Set 

Yes  3  No  9 

Comments: 

Pilot  2:  Raw  data  representation  doesn’t  give  clear  intuitive  picture  of 
required  corrections.  No  reference  for  angle  off  from  runway 
heading. 

Pilot  6:  Contrast  issue,  again  in  “No  Sensor”  cases. 

Pilot  1 0 :  Hard  to  judge  transition  to  land. 

Pilot  12:  Poor  pitch/vertical  decent  information. 
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The  Basic/BOTH  Symbology  Set 

Yes  2  No  10 
Comments: 

Pilot  2:  Confusion  factor. 

Pilot  6:  Contrast  in  “No  Sensor”  cases. 


Question  9.  What  strategy  did  you  incorporate  for  each  of  the  HUD  Symbology  Sets  in  the 
precision  approach  task? 

FuU  HUD 

Symbology  Set 

Pilot  1: 

Focusing  on  the  raw  data  ILS.  information,  the  depicted  wind 
component,  the  airspeed  readout,  and  the  flight  path  marker. 

Pilot  2: 

Work  hard  to  minimize  deviations  without  visual/sensor  image.  In 

VMC,  use  same  offset  to  see  runway. 

Pilot?: 

Ignore  many  items.  Concentrate  on  flight  path  vector.  Occasional  cross¬ 
check  at  airspeed. 

Pilot  4: 

Flew  raw  data  ILS. 

Pilot  5: 

Ignore  “VanHalen”  and  runway  symbol.  Fly  raw  data  and  flight  director 
commands. 

Pilot  7: 

Selectively  incorporate  only  the  symbols  that  provided  me  with  the 
information  I  wanted  into  my  cross-check. 

Pilot  8: 

Use  raw  data  for  azimuth  and  glidepath  (marker  cue  had  too  much  lag). 
Acceleration  and  speed  symbol  for  power. 

Pilot  9: 

Ignore  useless  symbology. 

Pilot  10: 

Mental:  ignore  clutter. 

Pilot  11: 

Don’t  allow  deviations  to  get  too  big.. .correct  as  soon  as  possible. 

Pilot  12: 

Full  pitch  steering  cue  and  monitor  alignment  with  runway  symbology. 

The  Basic/RWY  Symbology  Set 

Pilot  1 :  Wait  imtil  the  desired  landing  point  passes  between  the  manual  glidepath 
line  (the  dashed  line)  then  put  the  flight  path  indicator  on  that  desired 
landing  point.  Be  aware  of  actual  aircraft  heading  vs.  runway  heading. 

Pilot  2:  Very  intuitive  to  fly.  Normal  cross-check. 
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Pilots;  Work  hard. 

Pilot  4:  Flight  path  symbol  on  runway  symbol  at  3°  i 

Pilot  5 :  Strive  to  keep  the  runway  symbol  vertical  and  aimed  for  just  past  the 

approach  end  of  the  runway  symbol. 

Pilot  6:  Try  to  get  on  corrections  early. 

Pilot  7:  Pay  close  attention  to  3-D  location  (especially  GP)  using  available 
information. 

Pilot  8:  Keeping  the  FPM  where  I  wanted  to  land  on  the  runway  symbol  while 
checking  the  adjustable  horizontal  reference  with  runway  symbol. 
Cross-checking  airspeed. 

Pilot  9;  Line  up  the  dashed  “3  degree”  line  with  the  runway  symbol’s  “captain’s 
bars.” 

Pilot  10:  Used  visual  techniques  (which  by  the  way,  are  unreliable  in  IMC.) 

Pilot  1 1 :  Make  small  corrections  to  see  what  happened  if  I  didn’t  like  what  I 
saw... did  something  else. 

Pilot  12:  Stayed  on  left  edge  of  runway  symbol. 

The  Basic/RAW  Symbology  Set 

Pilot  1 :  Nail  down  the  course,  glidepath  and  airspeed,  then  center  the  flight  path 
indicator  in  the  crosshairs. 

Pilot  2:  Keep  errors  as  small  as  possible. 

Pilot  3 :  Fly  like  an  ILS. 

Pilot  4:  Raw  data  ILS. 

Pilot  5;  Position  flight  path  marker  to  keep  course  and  glideslope  deviation 
indicators  centered. 

Pilot  6:  Again,  get  on  corrections  early. 

Pilot  7:  Pretty  much  flew  it  like  an  F-1 6  approach. 

Pilot  8:  Flew  it  like  a  heads  down  ILS  with  a  standard  cross-check. 

Pilot  9;  Fly  it  like  a  regular  airplane! 

Pilot  10:  Standard  cross-check. 

Pilot  1 1 :  Small  corrections.  Keep  those  little  guys  centered. 

Pilot  12;  Stay  as  close  to  “on-track”  as  possible. 
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The  Basic/BOTH  Symbology  Set 

Pilot  1 :  Same  as  for  the  Basic  HUD  with  raw  data  only. 

Pilot  2:  Flew  the  runway  symbol  and  cross-checked  the  raw  data. 

Pilot  3 :  Fly  an  ILS  approach,  but  cheat  to  the  nmway  as  if  it  were  VMC. 

Pilot  4 :  Raw  data  IL  S . 

Pilots:  Flew  mostly  raw  data.  Started  using  the  runway  symbol  on  short  final 
to  line  up  for  touchdown. 

Pilot  6:  Look  at  it  like  a  “magic  eye”  picture.  The  3D  perspective  was  a  big 
help  when  added  to  the  raw  data. 

Pilot?;  No  strategy  necessary.  Fly  it  as  presented. 

Pilot  8:  Flew  like  a  heads-down  ILS  first  disregarding  runway  symbol  looking 
more  at  runway  symbol  below  500  ft  AGL. 

Pilot  9:  Fly  it  like  a  regular  airplane. 

Pilot  10:  Standard  cross-check. 

Pilot  1 1 :  Small  corrections.  Keep  those  little  guys  centered. 

Pilot  12:  Line  up  with  left  side  on  runway  symbol. 

Question  10.  Please  provide  any  additional  comments  or  design  alternatives  that  you  may 
have  regarding  each  of  the  HUD  Symbology  Sets. 

Full  HUD  Symbology  Set 

Pilot  1 :  The  HUD  concept  in  general  is  great!  I  like  looking  forward  to  where 
Fm  going  to  land  while  having  all  the  performance  feedback  right  in 
front  of  me  too.  Whatever  you  do,  keep  the  flight  path  indicator  and 
add  a  reference  ground  speed  in  just  below  actual  ground  speed. 

Pilot  2:  See  general  comment  sheet.  After  test  complete,  suggest  trying  Full 
HUD  minus  miniature  Aircraft,  ILS  raw  data  and  flight  direction 
symbol. 

Pilot  3 :  Have  color  options  for  better  contrast. 

Pilot  4:  For  the  limited  number  of  flights  accomplished,  I  was  not  at  the  level 
of  proficiency  that  I  could  appreciate  the  speed  worm  or  the 
acceleration  cue.  I  suspect  that  after  more  hours  of  experience,  I 
would  begin  to  notice  more  of  the  subtleties  of  the  system  such  as  this. 

Pilot  7 :  See  previous  suggestions.  Additionally,  heading  scale  is  too  large. 

You  could  shrink  it  down  a  little.  There  should  also  be  some  way  of 
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telling  the  pilot  that  ILS  information  is  present  and  being  monitored 
(e.g.,  glideslope  -  On;  localizer  -  On). 

Pilot  8:  Generally,  it  was  very  good.  Aircraft  display  (waterline)  was  too  large 
(see  questionnaire  for  diagram). 

Pilot  9:  In  the  F-16,  after  touchdown,  a  vertical  bar  appears  on  our  display 
depicting  runway  centerline.  In  CAT  III  conditions,  this  bar  is 
invisible.  I  suggest  replacing  your  vertical  dotted  line  with  a  centerline 
indicator. 

Pilot  1 0 :  Orange  color  for  night/IMC  without  sensor. 

Pilot  12:  See  questionnaire  for  diagram. 

The  Basic/RWY  Symbology  Set 

Pilot  1 :  By  far  the  easiest,  but  least  precise  of  the  HUD  depictions  to  use. 

Pilot  2:  Needs  course  indication  and  additional  performance  information,  but 
very  easy  to  fly. 

Pilots;  Good  pure  VMC  work. 

Pilot  7:  Some  features  need  to  be  added.  Reference  my  optimum  configuration 
on  the  videotape. 

Pilot  8:  Needs  raw  data  to  give  pilot  assurance  that  approach  is  safe. 

Pilot  10:  Use  orange  for  the  no  sensor  condition. 

Pilot  1 1 :  Needs  work. 

Pilot  12:  Delay  it  from  coming  on  until  (see  questionnaire  for  symbol  drawing) 
100  feet  AGL.  At  decision  height,  have  it  flash  3  or  4  times  then  go  out 
of  view.  This  will  prevent  continuing  the  approach  below  minimums. 

The  Basic/RAW  Symbology  Set 

Pilot  1 :  Really  nice  to  work  with  that  system!  The  best,  most  precise  and  in 
some  respects,  easiest  to  work  with. 

Pilot  2:  Hard  to  develop  a  mental  picture  of  relationship  of  course  to  raw  data 
where  the  flight  path  marker  should  be  moved  to  make  appropriate 
corrections. 

Pilot  3:  Good  starting  point. 

Pilot  8:  Interpreting  azimuth  (course)  was  initially  confusing... deciding  which 
way  to  steer  when  off  course.  Suggest  lateral  guidance  with  a  flight 
path  marker  cue. 
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Pilot  10:  Instead  of  raw  data,  use  flight  director. 

Pilot  12:  Workload  too  high. 

The  Basic/BOTH  Symbology  Set 
Pilot  2:  See  general  comments. 

Pilot  3 :  Near  perfect  configuration. 

Pilot  8:  Leave  runway  symbol  out  of  the  HUD  until  below  about  500  feet 

AGL. 

Pilot  9:  The  runway  symbol  is  a  nice  addition  to  the  new  data,  but  it’s  not 
necessary  when  a  radar  “sensor”  is  available  or  any  other  kind  of 
“actual”  runway  image. 

Pilot  10:  Orange  for  day  ILS,  needle/ball,  DME,  wind  and  flight  director. 
Pilot  12:  Delay  runway  symbol  and  eliminate  it  at  decision  height. 


Some  General  Questions 


The  HUD  Symbology  Sets 

Question  1.  Overall,  which  HUD  configuration  did  you  most  prefer?  Why? 


Table  C-25.  HUD  Symbology  Set  Preference  Frequencies 


Basic/RWY 

Basic/RAW 

Basic/BOTH 

Full 

Frequency 

1 

4 

4 

3 

Comments: 
Pilot  1: 

Pilot?: 

Pilots: 
Pilot  4: 
Pilots: 

Pilot  6: 
Pilot  7: 


Pilot  8: 

Pilot  9: 
Pilot  10: 
Pilot  11: 
Pilot  12: 


The  Basic  HUD  with  Raw  Deviation  Data.  Easiest  to  read  with  the 
most  precise  information. 

Runway  Symbol.  Clear,  easy  to  use,  uncluttered,  but  could  be 
improved. 

Basic  both  Runway  and  RAW. 

Basic  HUD  with  Raw  Deviation  Data. 

Almost  a  toss  up  between  Raw  and  Raw  with  runway.  These 
configurations  provided  the  right  mix  of  information  without  too  much 
clutter. 

Full  HUD... could  fine  tune  easier. 

Basic  HUD  with  both  Runway  and  Raw  Data... it  provided  the  best 
combination  of  information  without  too  much  cluttering  (there  was 
some). 

Full.  Although  it  was  a  bit  cluttered,  it  was  eerier  and  the 
speed/acceleration  cues  were  especially  helpful. 

Basic  HUD  with  raw  data.  For  me  it’s  the  easiest  display  to  fly. 

Both  -  maximum  data,  minimum  clutter. 

RAW  data  only.  All  information  in  there.  Is  similar  to  flying  ILS. 
Full.  Most  detail. 
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Question  2.  Overall,  which  HUD  configuration  did  you  least  prefer?  Why? 


Table  C-26.  The  Frequencies  for  Least  Preferred  HUD  Symbology  Set 


Basic/RWY 

Basic/RAW 

Basic/BOTH 

Full 

Frequency 

5 

1 

0 

6 

Comments: 

Pilot  1 :  The  Full  HUD.  Too  cluttered,  took  too  much  time  to  cross-check  and 
interpret  the  data. 

Pilot  2:  Full  HUD.  Too  much  clutter...even  interfered  with  VMC.  Very 
uncomfortable  in  touchdown  zone. 

Pilot  3 :  Basic  with  Runway. 

Pilot  4:  Full  HUD;  too  cluttered. 

Pilots:  Full  too  busy.  Often  got  symbols  confused. 

Pilot  6:  Rimway  only.. .too  subtle  on  cues. 

Pilot  7:  Full  HUD.. .too  much  information  made  it  cluttered  and  took  too  many 

brain  cells  to  interpret  all  of  the  presented  information. 

Pilot  8:  Basic  with  Runway  Symbol... it  looked  interesting,  but  does  not 

provide  pilot  with  enough  information  about  course  and  glideslope  for 
safety  until  close  in. 

Pilot  9:  Basic  HUD  with  Runway  Symbol. 

Pilot  10:  Full  too  cluttered. 

Pilot  1 1 :  Runway  only.  Harder  to  interpret  and  to  make  corrections.  Have 
feeling  something  is  missing,  but  it  apparently  works. 

Pilot  12:  Raw  Data.  Easy  to  confuse  data. 

Question  3.  Did  you  experience  any  problems  associated  with  the  design  and  mechanization 
of  any  of  the  individual  HUD  symbols?  If  so,  please  explain. 

Comments: 

Pilot  2:  Didn’t  fully  understand  relationship  of  course  and  runway  symbols 
until  after  test  was  over.  Rate  of  heading  tape  movement  very 
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Pilots: 

Pilot  5: 

Pilot  6; 

Pilot?: 

Pilot  8: 

Pilot  9: 

Pilot  10: 
Pilot  12: 


distracting.  Hard  to  get  peripheral  items  in  cross-check  because  they’re 
so  far  out.  Bank  pointer  and  miniature  AIRCRAFT  symbol  really  bad. 

The  Yaw  indicator  on  the  flight  director  was  only  cute... no  useful 
purpose.  The  flight  director  piper  tracked  with  the  aircraft  vector,  this 
was  confusing  at  times... require  second  order  interpretation.  Roll 
indicators  at  top  of  HUD  were  too  many  degrees  out-of-view...try  the 
bottom  half  instead. 

“VanHalen”  symbol  seemed  too  big  to  the  point  that  it  could  be  a 
distraction.  It  seems  its  primary  purpose  is  to  indicate  the  difference 
between  aircraft  heading  and  track  (for  crab  condition).  I  can  get  the 
same  information  from  my  heading  and  the  CDI.  Better,  yet,  it  might 
be  useful  to  add  a  “drift”  readout.  (For  example,  a  4°  R  drift  indication 
would  require  a  4°  L  crab.) 

Acceleration  caret,  speed  worm,  “GND”  symbol  on/near  touchdown. 
These  were  by  and  large  distractions.  Lack  of  DME  on  non-Full  BftJD 
cases. ..when  incorporated... on  Full  HUD  case. ..too  far  in  upper  left 
comer. 

The  flight  path  marker  needs  a  definite  reference  for  its  center  (see 
questionnaire  for  drawing).  The  flight  director  ball  should  be 
mechanized  to  do  both  azimuth  and  elevation  functions. 

Yes.  Aircraft  reference  symbol  was  too  large/too  apparent  for  its 
usefulness.  Suggest  a  single  line  (see  questionnaire  for  diagram). 
Flight  path  marker  cue  possessed  too  much  lag  and  did  not  provide 
lateral  guidance;  reduce  lag  and  provide  lateral  guidance. 

The  (see  questioimaire  for  drawing)  symbol  seemed  to  float  around 
aimlessly.  All  it  did  was  get  in  the  way  of  other  more  important 
symbols.  I  completely  ignored  the  vertical  dotted  lines,  and  lastly,  the 
small  circle  didn’t  provide  any  useful  information. 

Flag... useless;  small  circle... useless;  acceleration  cue... marginal  utility; 
speed  worm... marginal  utility;  aircraft  reference  bar... too  wide/thick. 

Aircraft  symbol  cluttered  data.  See  questionnaire  for  drawing. 
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The  TRAC  Simulator 


A.  Very  acceptable:  A  good  simulation  as  is. 

B.  Moderately  acceptable:  Minor  simulation  deficiencies  exist  that  do  not  impact  pilot 
performance. 

C.  Borderline:  Simulation  deficiencies  exist  that  could  impact  pilot  performance; 
improvements  would  be  desirable. 

D.  Moderately  unacceptable:  Simulation  deficiencies  degrade  pilot  performance; 
improvements  required. 

E.  Very  unacceptable:  Simulation  is  too  deficient  to  provide  the  intended  function. 


Question  1.  Flight  characteristics 


Table  C-27.  TRAC  Simulator  Flight  Characteristics  -  Rating  Frequencies 


Rating 

Frequency 

A 

3 

B 

6 

C 

1 

D 

2 

E 

0 

Comments: 

Pilot  8:  Yoke  ffeeplay  was  excessive!  Was  too  much  Dutch  roll! 
Pilot  12:  Directional  control  and  yaw  was  poor! 
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Question  2.  Throttle  characteristics _ 

Table  C-28.  TRAC  Simulator  Throttle  Characteristics  -  Rating  Frequencies 


Rating 

Frequency 

A 

3 

B 

6 

C 

1 

D 

2 

E 

0 

Comments: 

No  Comments  were  provided. 

Question  3.  Out-the-Window  display _ 

Table  C-29.  TRAC  Simulator  Out-the-Window  Display  -  Response  Frequencies 


Rating 

Frequency 

A 

7 

B 

3 

C 

2 

D 

0 

E 

0 

Comments: 

Pilot  7:  Limited  by  database  to  only  whole  degrees. 
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Question  4.  Sensor  image  representation _ 

Table  C-30.  TRAC  Simulator  Sensor  Representation  -  Rating  Frequencies 


Rating 

Frequency 

A 

? 

B 

4 

C 

1 

D 

0 

E 

0 

Comments: 

Pilot?:  Excellent. 

Question  5.  HUD  representation _ 

Table  C-31.  TRAC  Simulator  HUD  Representation  -  Rating  Frequencies 


Rating 

Frequency 

A 

6 

B 

5 

C 

1 

D 

0 

E 

0 

Comments: 

Pilot?:  Only  bad  case  is  WX  only. 
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Question  6.  Cockpit  geometry 


Table  C-32.  TRAC  Simulator  Cockpit  Geometry  -  Rating  Frequencies 


Rating 

Frequency 

A 

4 

B 

4 

C 

2 

D 

2 

E 

0 

Comments: 

Pilot  1 :  Throttle  quadrant  too  far  back. 
Pilot  7:  No  heads  down  stuff  to  reference. 


Question  7.  Were  you  able  to  adapt  to  the  simulator  aeromodel  in  a  reasonable  period? 
Yes _ No _ .  If  No,  please  comment. 


Yes  11  No  1 
Comments: 

No  comments  were  provided. 


Question  8.  Were  the  random  winds  adequate  at  inducing  a  nominal  amount  of  pilot  control 
in  order  to  remain  on  course  and  glideslope?  Yes  _ _ No _ . 


Yes  11  No  1 
Comments: 

Pilot  10:  Sinusoidal  variation  is  inappropriate... results  in  sustained  oscillation 
and  is  not  realistic. 


Question  9.  Was  the  Out-the-Window  scene  commensurate  with  your  experience  with 
approaches  in  low  visibility  conditions?  Yes _ No _ .  If  No,  please  comment. 


Yes  11  No  1 
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Comments: 


Pilot  2:  Visuals  okay,  but  lack  of  peripheral  vision  is  a  limitation. 
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The  ALG  Concept 


Question  1.  Do  you  think  there  is  a  future  for  ALG  in  the  major  commercial  passenger  and 
freight  industry?  Yes _ No _ .  Comments; 

Yes  11  No  1 
Comments: 

Pilot  3 :  More  likely  to  be  automatic  and  autonomous. 

Pilot  4:  I  don’t  think  it  will  replace  CAT  III  auto-land  approaches. 

Pilot  8:  Absolutely. 

Pilot  9:  Although  auto-pilot  coupled  GPS  approaches  seems  to  be  where  we’re 
headed,  some  sort  of  heads-up  “sensor”  image  would  implement  this 
system  perfectly. 

Pilot  1 0:  Best  thing  since  instruments.. .fighters  (F-1 1 1)  had  it.. .awesome 
capability. 

Pilot  12:  Add:  Angle-of-path  information  and  sell  a  “windshear/energy 
management”  program. 

Question  2.  Using  an  ALG  display  similar  to  the  one  in  this  evaluation,  would  you  be 
comfortable  flying  low  visibility  approaches?  Yes _ No _ .  Comments: 

Yes  10  No  2 
Comments: 

Pilot  2:  Display  improvement  required.  Sensor  image  adequate  if  it  can  achieve 
that  fidelity  in  all  WX  conditions. 

Pilot  3 :  I’m  never  comfortable  flying  in  low  visibility  situations! 

Pilot  4;  Particularly  for  a  coupled  approach,  I  think  the  HUD  would  enhance 
safety. 

Pilot  7:  With  the  modifications  to  the  HUD  to  reduce  clutter. 

Pilot  1 1 :  Am  I  flying  or  monitoring  the  auto  pilot? 
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General  Comment  Sheet 


Pilot  2:  Placement  of  bank  pointer  is  distracting.  When  flying  the  PPM  as  primary,  you 
have  to  search  for  pointer.  Doesn’t  help  that  it  is  a  long  way  off 


The  course  indicator  is  useless  and  a  hindrance  when  flying  ILS  approach.  With  the 
HSI  implementation,  the  course  indicator  gives  feel  for  orientation  to  nmway,  but 
doesn’t  observe  central  area  needed  for  deviation  indication.  The  HUD 
implementation  doesn’t  give  feel  for  orientation.. .only  whether  right  or  left. 


The  miniature  aircraft  symbol  is  way  too  large  and  (poorly  shaped)  to  enhance 
FPM-based  system. 


Lag  in  flight  director  cue  is  unacceptable.  Doesn’t  appear  to  be  just  lag  in  some 
cases,  but  extremely  slow  movement  rate  or  “gain”  to  make  adequate  correction. 


How  is  the  horizon/heading  tape  implemented?  The  pitch  sensitivity  is  very  good, 
but  the  heading  constantly  sliding  by  at  high  rate  is  very  distracting. 


The  full  display  is  much  too  cluttered  for  normal  VFR  operation. 


The  runway  symbol  perspective  is  minimal  and  barely  effective  at  preventing 
anything  during  final  approach. 


Radar  ALT  is  in  useful  position.  Airspeed  and  ALT  too  far  from  center. 


Comer  reflector  presentation  helps  with  perspective. 


Using  the  steering  bar  presentation  with  the  glidepath/touchdown  zone  presentation 
seems  awkward  to  me.  I  think  there  may  be  a  way  to  optimize  the  glidepath 
touchdown  zone  presentation  with  a  similar  lateral  presentation  to  be  overall  more 
consistent  with  T-PM. 
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See  questionnaire  for  drawings.  This  position  is  a  wag  with  the  current  symbology 
because  the  dotted  lines  aren’t  there.  If  they  were  a  periphery  or  center  with 
opposite  location  containing  target  values,  then  computation  would  be  automatic. 
With  good  enough  perspective  on  runway  symbol  (and  extended  center  line)  the 
CDI  approach  might  not  be  needed  at  all. 
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